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Executive Summary 


Zimpher’s goal is to ensure every SUNY student has the op- 

portunity to engage in at least one form of applied learning 
before they graduate. Applied learning, or developing skills and 
knowledge from direct experiences outside the classroom, varies 
widely across university systems, within campuses, and between 
programs. SUNY’s applied learning initiatives include 
work-based activities, e.g., co-ops, internships, work study, and 
clinical placement (SUNY Works); community-based activities, 
e.g., service learning, community service, and civic engagement 
(SUNY Serves); and discovery-based activities, e.g., research, en- 
trepreneurship, field study, and study abroad (SUNY Discovers). 
Our report focuses on the work-based activities of SUNY Works. 

With funds from the Carnegie Corporation of New York, 

SUNY asked the Rockefeller Institute of Government to examine 
applied work-based experiences of SUNY Works along two key 
dimensions. First, situate SUNY Works relative to other applied 
learning initiatives by asking: What do other states, systems, and 
countries do? What can SUNY learn from the experiences elsewhere that 
is relevant for extending work-based applied learning opportunities more 
widely? Second, because expanding work-based applied learning 
experiences includes the implicit goal of expanding worthwhile 
opportunities for students, how can SUNY understand the effects of 
such experiences on the retention and graduation of SUNY students and 
employment of SUNY graduates? The report addresses these two di- 
mensions in depth. 


S tate University of New York (SUNY) Chancellor Nancy 


Situating SUNY Works 


This report describes relevant experiences of campuses and 
systems elsewhere, identifying what SUNY Works should con- 
sider to extend work-based applied learning to more students on 
each SUNY campus and across the system as a whole. However, 
none of the campuses or systems we examine is a direct comparison to 
SUNY because SUNY Works is unique: there is no other state or system 
in the US that has advanced a work-based learning experience initiative 
on a scale and across the breadth of types of study programs and institu- 
tions encompassed by SUNY. SUNY’s uniqueness is due, in part, to 
its reach— across campuses that encompass all types of institu- 
tions, levels of degree program, and fields. 

Although there is no domestic system initiative quite like 
SUNY’s, drawing on field experts and selected literature, we iden- 
tify key features of other system-wide and campus-level initiatives and 
suggest that these are the relevant features for SUNY to consider as it 
expands SUNY Works. These features are important components of 
seemingly successful or recent initiatives and may be used as a ba- 
sis of comparison. In particular, we suggest that successful 
work-based applied learning opportunities: 
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@ are well-structured. There are clear expectations for stu- 
dents and stakeholders as well as sufficient administrative 
support. Organized workshops and/or seminars are pro- 
vided on campus for participating students to introduce 
them to the basic language, skills, and expectations in the 
field in which they are working. 


™ foster opportunities for applied learning experiences 
throughout the degree program. Students can draw from 
and relate to such experiences throughout the academic 
curriculum, building knowledge and capacities over time. 


m™ are credit bearing to recognize the time and effort that stu- 
dents and faculty dedicate to the experience and to allow 
administrative tracking and oversight of applied learning 
experiences. 


™ incorporate alternative assessments of student learning as 
part of the experience (e.g., noncognitive, prior-learning 
assessment (PLA), portfolio). 


To show how work-based experiences can be scaled up across a cam- 
pus and through a system we highlight points of leverage to engage stu- 
dents, faculty, and employers in applied learning initiatives. Drawing 
from the same knowledge base, we identify as points of leverage, 
operating at system or campus levels: 


™ mandates that require campuses to allow for, or students 
to engage in, learning outside the classroom; 


™ financing to support faculty, students, and/or employers 
engaged in work-based learning experiences; 


™ supportive arrangements and infrastructure, to facilitate 
participation of faculty, students, and/or employers. 


We illustrate how the features of initiatives are manifested 
and points of leverage work by examining in detail five cases of 
initiatives that extend (or intend to extend) opportunities for 
work-based applied learning broadly, including three sys- 
tem-wide initiatives (Australia’s learning and teaching policy ini- 
tiatives; the Dual System in Germany; and the Universities of 
Applied Sciences in Switzerland) and two campus-wide initiatives 
(Northeastern University’s co-op experience and the University of 
Michigan’s Undergraduate Research Opportunity program). 
These five illustrative cases show not every initiative includes ev- 
ery feature, and the particular points of leverage adopted address 
the needs of academic programs, students, faculty, and commu- 
nity stakeholders (prominently, employers) concerned. 

The cases are supplemented by more than two dozen exam- 
ples of relevant applied learning initiatives or broad-based strate- 
gies. We show how systems and campuses have used mandates to 
encourage student engagement outside the classroom (University 
System of Maryland, Youth Guarantee Finland); financial incen- 
tives to encourage employer-partnerships (Trade Adjustment As- 
sistance Community College and Career Training) and to facilitate 
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expansion of work-based learning opportunities at campuses 
(BrownConnect, Engaged Cornell); and infrastructure support to 
generate buy-in from faculty (Michigan’s Urop), students (Cal 
Poly), and employers (South Dakota School of Mines). Deep com- 
mitment to work-based applied learning at places like Northeast- 
ern, Berry, or Berea means that work is fully integrated into the 
core educational curriculum and mission of these institutions, 
making these models of integrated work-based applied learning. 
Although useful, such models may be difficult to introduce else- 
where without substantial infrastructure support and deeper 
changes in mission and reward systems. Implementing meaning - 
ful work-based applied learning experiences across a campus or 
system is hard to do, requiring agreements and engagements that 
span conventional arrangements. Yet, these and other identified 
experiences provide examples of the kinds of features and points 
of leverage that could figure in strategies developed by SUNY 
Works. 


Understanding Effects 

Taking initiatives to scale requires a basic understanding of 
what works. We conduct a feasibility study to evaluate the use of ad- 
ministrative data to measure effects of work-based applied learning expe- 
riences. With support from the vice provost for institutional 
research and officials at one campus in the SUNY system, our pi- 
lot study explores the possibility of using campus academic unit 
records to generate estimates of the effects of internships — one 
form of work-based applied learning — on retention and gradua- 
tion. The pilot study also explores the use of linked academic — 
NYS Department of Labor Unemployment Insurance (UI) wage 
records to generate estimates of the effects of internships on em- 
ployment outcomes. We find that the effects of internships on reten- 
tion and graduation can be estimated with academic unit records. In our 
pilot study, we identify limitations and considerations that apply to such 
efforts, noting that student records may lack information on internships 
needed to permit detailed and comprehensive analyses. The initial and 
tentative findings suggest positive effects: internships are associ- 
ated with better retention and graduation rates. The link with 
wage records, at present, requires steps to address certain restric- 
tions imposed by the NYS Department of Labor. While we de- 
scribe the design, measures (their limitations and interpretation), 
and implementation of data transfer and linkage to date, we are 
not yet able to report results on employment outcomes. 


Conclusion 

There are no work-based applied learning initiatives in the US 
on par with SUNY’s. Nevertheless, we examine relevant initia- 
tives and strategies adopted by other systems and campuses to 
identify and illustrate the key features of work-based initiatives 
and the main strategies for scaling up relevant to SUNY. 
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SUNY, on some campuses and for some fields, incorporates 
the features and uses points of leverage discussed above: 
work-based applied learning initiatives on SUNY campuses often 
are well-structured and include organized workshops. But other 
features appear to be largely absent: work-based applied learning 
opportunities tend not to be offered early in the student’s aca- 
demic career, are not uniformly well-integrated into the curricu- 
lum, and do not include alternative assessments. Further, 
participation in work-based applied learning varies across many 
campuses, often by field of study. Of course, experience elsewhere 
shows that not all of the identified features are present in 
work-based applied learning initiatives that have been extended, 
or are being extended, to all students. Moreover, depending on 
the field and student interests, the benefits of applied learning ex- 
periences may come from service learning and discovery learning 
as much as from work-based learning. The effects on retention, 
graduation, and employment of differences in the types and fea- 
tures of applied learning, field of study, and student background 
warrant closer study using administrative academic and wage re- 
cords or other methods (e.g., experimentation). 

We suggest, however, that the experience elsewhere shows 
successful and promising work-based applied learning initiatives 
that reach large proportions of students. The features of those ini- 
tiatives and the means used to leverage and support applied 
learning experiences within those initiatives reveal possible op- 
tions for consideration by SUNY Works as it seeks to bring 
work-based applied learning to scale — on campuses and across 
the system. 
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Introduction 


ory whereby students develop skills and knowledge from 

direct experiences outside a traditional classroom setting.! 
In practice, applied learning includes initiatives from internships 
and co-ops to civic engagement and service learning.” It can benefit 
students by bridging the gap between theoretical knowledge and 
its practical application, increasing self-confidence, and develop- 
ing critical thinking and career goals. 

Applied learning at the State University of New York (SUNY) 
covers a wide range of experiential learning. SUNY’s own taxon- 
omy refers to three main categories: work-based activities, like 
co-ops, internships, work study, and clinical placements, under 
SUNY Works; community-based activities, such as service learn- 
ing, community service, and civic engagement, under SUNY 
Serves; and discovery-based activities, like research, entrepreneur- 
ship, field study, and study abroad, under SUNY Discovers. This 
report examines work-based initiatives coming under SUNY 
Works. Even here, the types of work-based experiences differ in 
nature and structure. 

In 2012, with support from Lumina Foundation as well as the 
Carnegie Corporation of New York, SUNY launched “SUNY 


A pplied learning is the application of previously learned the- 
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Works.” The overarching goal of the initiative was to develop a 
model internship and cooperative education infrastructure to help 
bring higher education in line with labor market needs. The initia- 
tive’s initial objective in 2012 was to use cooperative education to 
decrease the time to graduation for adult learners.4 Funds from 
the Carnegie Corporation of New York were earmarked for 
scale-up of the initiative. 

Through new and expanded internships and cooperative 
learning experiences across eighteen SUNY campuses, SUNY 
Works aims to increase student retention and degree completion 
and to improve graduate employment outcomes.° By providing 
both paid and unpaid work experiences to students, SUNY Works 
helps make “academics more relevant by connecting students to 
work and integrating work experience with curriculum.” Paid ex- 
periences in particular can also help to offset the financial pres- 
sures many students face. 

Lumina monies were used to assess SUNY Works over the 
short and medium term. A first evaluation, carried out by the 
Rockefeller Institute of Government, assessed experiences at nine 
SUNY campuses, collectively referred to as Phase I pilot sites.” 
This report found that while Phase I schools were developing 
model co-op and internship initiatives consistent with the values 
and commitments of their respective institutions, the success of 
these campus-level initiatives was contingent on SUNY’s ability to 
better integrate applied learning into the curriculum and to help 
increase participation among students, faculty, and employers 
alike. The report concluded with a recommendation to evaluate 
the effects of SUNY Works and other forms of applied learning, 
with a particular focus on student graduation, retention, and em- 
ployment rates. As part of a Carnegie Corporation of New York 
grant, the present study builds on the earlier report in two ways. 

First, this report situates SUNY Works against relevant fea- 
tures of applied learning and strategies used to extend such learn- 
ing experiences more widely (points of leverage) in other states 
and systems. We identify those initiative features and points of leverage 
where the experience has particular relevance for campus-wide, re- 
gion-wide, or SUNY-wide expansion of SUNY Works. We intention- 
ally focused on the system and campus levels (as opposed to 
academic program level) because these levels reflect the reach of 
the vision for applied learning at SUNY. The results of this com- 
parison suggest that, although SUNY Works is unique in the 
United States (we found no other state or system that has ad- 
vanced a work-based learning experience initiative on a scale and 
across the breadth of types of study program and institution en- 
compassed by SUNY Works), we can provide useful comparisons 
with other initiatives by focusing on components of these endeav- 
ors, particularly features of existing initiatives and the points of le- 
verage used to support or bring these initiatives to scale. We 
generated these features and leverage points through discussions 
with experts and reference to selected literature. 
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Second, because the explicit goal of expanding SUNY Works 
includes an implicit goal of providing worthwhile opportunities 
for students, we carried out a feasibility study to evaluate the use 
of administrative data to measure effects of work-based applied 
learning experiences on one campus in the SUNY system. The pi- 
lot study explores the possibility of using campus academic unit 
records to generate estimates of the effects of internships on reten- 
tion and graduation. The pilot study also explores the use of 
linked academic-NYS Department of Labor Unemployment Insur- 
ance (UI) wage records to generate estimates of the effects of in- 
ternships on employment outcomes, an information base for 
analysis of higher education performance now used in many 
states. For campuses that collect and maintain data on internships, ad- 
ministrative data can uncover effects on retention and graduation. The 
analysis using wage record information, at present, requires steps to ad- 
dress certain restrictions imposed by the NYS Department of Labor. The 
report discusses both the opportunities and limitations when it co- 
mes to design, measurement, implementation of data transfer and 
linkages, and analysis and interpretation of results. 

These two dimensions are accomplished in a report with four 
parts. Part I contains a brief description of the approach adopted 
in this study. In Part II, we identify relevant approaches and expe- 
riences elsewhere to help situate applied learning coming under 
SUNY Works. Part III includes the account of the pilot study of 
the use of administrative data to estimate the effects of 
work-based applied learning at one SUNY campus. A short sum- 
mary with conclusions is presented in Part IV. The report has ad- 
ditional details in the appendices, which can be consulted for 
more information. 
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Part I: Study Approach and Design 


Given study and time constraints, we adopted an approach for 
the project judged to be most promising. The inquiry followed 
three strands. 

First, to gain a deeper understanding of the history, directions, 
and decision-making for applied learning on SUNY campuses, we 
designed a survey instrument to collect information from the pool 
of campus representatives interested or involved in applied learn- 
ing. The instrument is provided in Appendix A. Participants at- 
tending the September 18-19, 2014, SUNY Applied Learning 
Workshop in Syracuse, New York, comprised the target popula- 
tion; all participants were invited by Elise Newkirk-Kotfila to 
complete the survey. Although relatively few participated in the 
survey, their responses along with information available from 
other campus-level surveys carried out by SUNY enable a descrip- 
tion of features and coverage of learning experiences coming un- 
der applied learning initiatives at SUNY campuses. 

Second, to uncover relevant experience elsewhere, we engaged 
both current field experts and contemporary literature. We con- 
sulted experts knowledgeable in the field of campus-wide and sys- 
tem initiatives for work-based applied learning, other applied 
learning, or curriculum development and implementation more 
generally.§ Project staff participated in three conferences featuring 
presentations on current initiatives and experiences relevant to this 
study.’ Project staff also conducted a review of extant research, doc- 
uments, or other materials on approaches to and experiences with 
applied learning or curriculum change, again at system or campus 
level. From this examination, we employed a framework developed 
for this project that identifies key features of applied learning initia- 
tives and the points of leverage which can be used to support or 
bring about opportunities for students across campuses and the 
system.!0 The features and points of leverage help to situate SUNY 
Works against experience elsewhere. They also point to approaches 
in use that might be considered in the development and refinement 
of strategies in SUNY Works. The examples provided here are gen- 
erated from the best information available to us, within the 
timeframe and constraints of the project. 

Third, to understand in more detail if it is possible for SUNY 
or SUNY campuses to use administrative data to evaluate their 
initiatives we carried out a pilot study of the effects of work-based 
applied learning. Undertaken with advice from officials at one 
SUNY campus and the SUNY vice provost for institutional re- 
search, the pilot study seeks to evaluate a method for using avail- 
able campus academic unit records linked to NYS Department of 
Labor UI wage records to gauge the effects on retention, gradua- 
tion, and employment rates associated with participation in in- 
ternships. The pilot allowed for a test of the feasibility of such a 
method in terms of the analysis and metrics, timeliness, imple- 
mentation requirements, and usefulness of findings. 
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Part Il. Situating SUNY Works 


A: Grounds for Comparison 

SUNY Works promotes work-based learning designed to im- 
part career readiness in students and improve college retention 
and completion rates. Initially, the aim of the initiative was to fa- 
cilitate paid, credit-bearing experiences in fields with high em- 
ployment needs such as science, technology, engineering, and 
math (STEM) fields." As the initiative has continued to grow, 
however, the emphasis on particular fields and learners has given 
way to an expansion to all SUNY students. Consistent with the 
goal of providing opportunities for every student to have an ap- 
plied learning experience, SUNY Chancellor Nancy Zimpher, 
alongside a newly comprised task force of business leaders, 
pledged to engage the CEOs of Fortune 500 companies through- 
out New York, as well as other large employers, to secure their 
participation and involvement in the initiative. Already, some of 
the state’s largest companies including GLOBALFOUNDRIES, 
General Electric, and IBM have agreed to partner with SUNY 
schools, alongside local companies like Welch Allyn, Inc., a medi- 
cal device manufacturer with unique employment needs. '” 

The scope of work-based learning initiatives of SUNY Works 
has no equivalent. We have not found a counterpart, under 
state-wide initiative, that envisions broad participation for stu- 
dents across such a range of campuses and fields. Not surpris- 
ingly, the work-based applied learning experiences themselves 
vary. They may be paid or unpaid, credit bearing or noncredit 
bearing,!° completed during the school year or the summer, with 
or without faculty supervision. 

Our approach is to situate SUNY Works against a range of ex- 
perience elsewhere relevant to SUNY’s goal of extending 
work-based applied learning opportunities to all students, across 
fields of study or across campuses. Because SUNY is a 
sixty-four-campus system that includes community colleges, tech- 
nical programs, four year colleges, and research universities, our 
range of institutional initiatives is similarly broad. We look at ex- 
periences in systems, like Australia, which recently has sought to 
extend work-based applied learning in both universities and 
Technical and Further Education (TAFE) institutes; community col- 
lege sector initiatives, like the Trade Adjustment Assistance Com- 
munity College and Career Training (TAACCCT) Grants, which 
employ external funding to bring about multicampus/ 
multiemployer partnerships for skill training; four-year sector ini- 
tiatives, like the University System of Maryland, which mandates 
students to earn 10 percent of credits outside of the classroom, in- 
cluding internships; private four-year campuses like Northeastern, 
which has well-integrated work-based experiences; and public uni- 
versities, like the University of Michigan, which through its fac- 
ulty-led Undergraduate Research Opportunity Program, engages 
students from all fields in applied learning experiences. In total, 
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we present relevant approaches and experiences of more than two 
dozen systems, sectors, campuses, and other levels or programs of 
learning, both domestic and international. We refer to these exam- 
ples throughout the report. Table 1 lists the initiatives by level 
(system through campus) and type of provider (e.g., public 
four-year) that we examined. 


Table 1: Level of Initiative and Type of Provider 


(in Place or Under Discussion for Eventual Implementation) 


Systems 
Australia (System-wide strategies) 
Finland (Youth Guarantee) 
United States (Federal Work Study) 
Massachusetts (civic learning policy) 
North Dakota (in-state internship) 
Washington (State Work Study) 
Indiana (State Work Study) 
Illinois (State Work Study) 


Sector Two-Year 
and Career Training (TAACCCT) 


development) 


Sector Four-Year 


Efficiency initiative) 
Universities of Applied Sciences, 
Switzerland 


Private Four-Year Campuses 
Berea, KY (Learning, Labor & Service) 
Berry, GA (Work Experience) 
Brown (Brown Connect) 
Cornell (Engaged Cornell) 
Drake 
Duke 
Northeastern (Cooperative Education) 


Trade Adjustment Assistance Community College 
Technical College System of Georgia (workforce 


Connecticut (integrated, paid internship) 


University System Maryland (Effectiveness and Dual System, Germany (Apprenticeship) 


Public Four-Year Campuses 


California Polytechnic State University (Learn by 
Doing) 


California State University, Monterey Bay (service 
learning) 


University of Michigan (UROP) 
University of Texas (applied research) 
South Dakota School of Mines (work experience) 


Royal Melbourne Institute, Australia (work 
integrated learning; pathways) 


Other Worked Based 


Council for the Assessment of Experiential Learning 
(Learning Assessments) 


Educational Testing Service (“Fitness for Work” 
Assessment) 


Tennessee Board of Regents (High Impact Practices) 


Photovoltaic Manufacturing Consortium (with 
SUNY Poly) 


South Carolina (Apprenticeship) 


Rockefeller Institute 


While we illustrate a range of systems, sectors, campuses, and 
other settings for work-based applied learning, these venues, like- 
wise, offer a variety of applied learning experiences. First, we 
mapped the initiatives (outlined in Table 1) against type of experi- 
ence (e.g., internship, coop) and for whom (all fields, targeted) 
they are offered. Table 2 provides a bird’s eye view of more than 
two dozen applied learning initiatives identified and examined. It 
situates SUNY Applied Learning and SUNY Works against these 
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initiatives. Some initiatives, like Australia’s policy on teaching and 
learning, suggest a good comparison with SUNY Works: both are 
systems, both are comprised of internships and coops, both are 
not restricted by field or targeted to a particular group of stu- 
dents. Indeed, we offer an extensive description of the relevant 
Australian strategies. Other initiatives, like University of Michi- 
gan’s Undergraduate Research Opportunity program, have key 
differences from SUNY Works: Urop is research- not work-based. 
However, the initiative features and scale-up have been innova- 
tive and demonstrated results, allowing SUNY the opportunity to 
draw from the experience lessons applicable to its own 
work-based applied learning initiatives. 

Second, we mapped the initiatives against initiative features that we 
have identified as “good” or “effective” or “relevant” learning in the lit- 
erature and among experts.'4 Such features include experiences 
that: 


™ are well-structured. There are clear expectations for stu- 
dents and stakeholders as well as sufficient administrative 
support. 


™ foster opportunities for applied learning experiences 
throughout the degree program. Students can draw from 
and relate to such experiences throughout the academic 
curriculum, building knowledge and capacities over time. 


™ include organized workshops and/or seminars on campus 
for participating students to introduce them to the basic 
language, skills, and expectations in the field in which they 
are working. 


M™ are credit bearing to recognize the time and effort that stu- 
dents and faculty dedicate to the experience and to allow 
administrative tracking and oversight of applied learning 
experiences. 


™ incorporate alternative assessments of student learning as 
part of the experience (e.g., noncognitive, prior-learning 
assessment (PLA), portfolio). 


The list here is intended to be illustrative rather than a check 
list for an ideal type. Successful initiatives, detailed below, use 
some combination of these key features, often in very different 
ways. Even limiting the effects of interest to retention, graduation 
and eventual employment, work-based applied learning experi- 
ences coming under SUNY Works are likely to be more effective 
when they are well-designed; supported by site supervisors and 
faculty; and assessed in ways that serve student, employer, and 
academic interests. 

Table 3 situates SUNY and SUNY Works with respect to fea- 
tures that are judged to foster “good” or “effective” or “relevant” 
learning. Yet SUNY’s applied learning initiatives, in general, and 
SUNY Works, in particular, are not single system-wide initiatives. 
Instead, they encompass a collection of practices across a range of 
campuses. Because SUNY does not mandate particular features 
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Table 3. Effective Initiative Features 


Fully Early/ 
Integrated/ Sustained Workshops/ Credit Alternative 
Well Applied Seminars Bearing Assessments 
Structured learning 
SUNY Applied Learning Initiative x x x 
SUNY Works x x x 
System Level 
Australia x x x 
Finland 
United States x 
Massachusetts x 
North Dakota x 
Washington x 
Indiana x 
Illinois x 
Sector Two-Year 
TAACCCT x x x x 
Technical College System, GA 
Connecticut x 
Sector Four-Year 
University System of Maryland x x x 
Universities of Applied x x x x x 
Sciences, Switzerland 
Campus, Private Four-Year 
Berea, KY x x x x x 
Berry, GA x x x x x 
Brown 
Cornell x x x x 
Drake x x x 
Duke 
Northeastern x x x x x 
Campus, Public Four-Year 
California State Polytechnic x x x x x 
Institute 
California State University, x x x 
Monterey Bay 
South Dakota School of Mines x x x x 
Royal Melbourne Institute of x x x x 
Technology, Australia 
University of Michigan x x x x x 
University of Texas x 
Other Work Based 
Council for the Assessment of x 
Experiential Learning (CAEL) 
ETS, “Fitness for Work” x 
Tennessee Board of Regents x 
Dual System, Germany x x x x x 
South Carolina x 
Photovoltaic Manufacturing x x 
Consortium (with SUNY Poly) 
Note: “x” indicates the presence of a particular initiative characteristic, feature and/or point of leverage 
as indicated by the column header. Cells not marked with an “x” indicate the initiative feature either does 
not apply or fell outside the purview of our analysis. 
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and because, even if it did, there would still be variation in what 
happens in practice, the rows for SUNY and SUNY Works reflect 
general trends observed at some pilot campuses for some fields of 
study rather than a strategy SUNY has put in place to foster the 
implementation of such features system-wide. For example, some 
Phase I and Phase II schools routinely conduct “workshops” 
while others do not. 

Third, the approaches and experiences reviewed reveal strate- 
gies and approaches used to bring about change in learning and teach- 
ing, some specifically with reference to applied learning experi- 
ences. Although change, whether building on existing provision 
or introducing new provisions, may naturally follow the interests 
of those most directly involved — including students, faculty, or 
employers — systems or campuses, through incentives, informa- 
tion, or mandates may also help to bring it about.15 

For scaling up across a campus and throughout a system, 
strategies may use points of leverage to engage students, employ- 
ers, and faculty in work-based applied learning. Summarized in 
brief, leverage points include: 

With respect to students: 


® workshops/seminars that introduce students to the idea of 
applied learning and support them when they are partici- 
pating in it; 

™ an attractive opportunity for work-based learning that pre- 
pares students for employment; 


™ incorporation of work-based application for self-assess- 
ment and reflection; 


™ academic credit for work-based learning experiences 
and/or a stipend/wage. 


With respect to employers 


™ subsidies to share cost of stipends (e.g., through Federal 
Work Study); 


™ support for employer effort (design of internship, job pro- 
files, informational support like a 1-800 number for inter- 
ested employers); 


™ atrue “hiring” process that permits employers to select 
students; 


™ enabling employers to work through groups, e.g., consor- 
tia, associations (especially for small and medium size en- 
terprises); also strengthening overall community 
engagement, to develop good will and to make case for 
“win-win” for work-based applied learning; 


™ ameans to track and approach employers of current stu- 
dents and graduates, to establish and sustain campus-em- 
ployer relationship. 


With respect to faculty: 


m™ directed funding for work-based applied learning; 
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m@ allocation of effort to internship or co-op learning, such as 
heading a student workshop; 


™ recognition for faculty effort, ranging from selective 
named awards and salary supplements to weight in pro- 
motion and pay systems; 


™ mandating applied learning experience special targeting of 
work-based applied learning opportunities to improve stu- 
dent success (responds to a disposition of faculty for 
underrepresented students to succeed). 


We discuss and illustrate many of these leverage points in Part 
II below. As noted there, a number of these points of leverage al- 
ready are in place across SUNY. However, information is limited 
on the number of campuses using any one (or more) of these 
means and, if used, the present coverage by field or type of stu- 
dent. More generally, the list invites consideration of additional 
points of leverage that might be well-suited to extend work-based 
learning experiences more widely, at campus or system levels. 

Table 4 locates each identified approach or experience with re- 
spect to its points of leverage as they relate to students (4a), em- 
ployers (4b), and faculty (4c). Again, rows for SUNY and SUNY 
Works represent trends at the pilot campuses. For example, many 
pilot campuses have “useful assessments” in place (Table 4a), but 
some are still struggling to develop measurement tools. 

Although, for analytical purposes, we separate features and 
points of leverage, in practice they go hand-in-hand in creating 
successful initiatives: Worthwhile work-based applied learning 
experiences are developed, improved, and sustained through sup- 
port and requirements. 

In the next two sections, we describe both key features and le- 
verage points found in five illustrative cases and more than a 
dozen examples. The cases put the system- or campus-level expe- 
riences with work-based applied learning in context. The exam- 
ples that follow aim primarily to direct attention to concrete 
means — points of leverage — used to bring about or sustain 
widely provided opportunities for applied learning at system and 
campus levels. 


B. Illustrative Cases 


While not an exact match to SUNY, the five cases offer impor- 
tant takeaways for SUNY as it seeks to expand work-based ap- 
plied learning experiences. As mentioned above, Australia’s 
Commonwealth higher education policy comes closest to a sys- 
temic approach that incorporates both universities and technical 
programs. In addition to Australia, we illustrate Germany’s dual 
system and Switzerland’s Universities of Applied Sciences, which 
covers a sub-set of the levels or sectors coming under SUNY 
Works. Because SUNY’s approach is often manifested in bot- 
tom-up initiatives, we also feature campus-wide initiatives: 
Northeastern University’s co-op experience, an initiative built into 
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Table 4a. Points of Leverage for Student Participation 


Workshops Relevant Useful or 
/Seminars placements Assessments Credit 
SUNY Applied Learning Initiative x x x x 
SUNY Works x x x x 
System Level 
Australia x x x x 
Finland x 
United States x x 
Massachusetts 
North Dakota x x 
Washington x x 
India x x 
Illinois x x 
Sector Two-Year 
TAACCCT x x x x 
Technical College System, GA x x 
Connecticut x x 
Sector Four-Year 
University System of Maryland x 
Universities of Applied x x x x 
Sciences, Switzerland 
Campus, Private Four-Year 
Berea, KY x x x x 
Berry, GA x x x x 
Brown x x 
Cornell x x x x 
Drake x x x 
Duke x 
Northeastern x x x x 
Campus, Public Four-Year 
California State Polytechnic x x x x 
Institute 
California State University, x x 
Monterey Bay 
South Dakota School of Mines x x x 
Royal Melbourne Institute of x x x x 
Technology, Australia 
University of Michigan x x x x 
University of Texas x x 
Other Work Based 
Council for the Assessment of x 
Experiential Learning (CAEL) 
ETS, “Fitness for Work” x 
Tennessee Board of Regents 
Dual System, Germany x x x x 
South Carolina x x 
Photovoltaic Manufacturing x x x 
Consortium (with SUNY Poly) 
Note: “x” indicates the presence of a particular initiative characteristic, feature and/or point of leverage 
as indicated by the column header. Cells not marked with an “x” indicate the initiative feature either does 
not apply or fell outside the purview of our analysis. 
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Table 4b. Points of Leverage for Employer Participation 


Work 
Ce Geer pee Eoplye:  veuuarne 
Groups 
SUNY Applied Learning Initiative x x 
SUNY Works x x 
System Level 
Australia x x x x x 
Finland 
United States x x x 
Massachusetts 
North Dakota x x x 
Washington x x x 
India x x x 
Illinois x x x 
Sector Two-Year 
TAACCCT x x x x 
Technical College System, GA x x x x 
Connecticut x x x x 
Sector Four-Year 
University System of Maryland 
Universities of Applied x x x x 
Sciences, Switzerland 
Campus, Private Four-Year 
Berea, KY x x x x x 
Berry, GA x x x x x 
Brown x x x x 
Cornell x x x x 
Drake x x x x 
Duke x x x x 
Northeastern x x x x x 
Campus, Public Four-Year 
California State Polytechnic x x x x 
Institute 
California State University, x 
Monterey Bay 
South Dakota School of Mines x x x x x 
Royal Melbourne Institute of x x x x x 
Technology, Australia 
University of Michigan x x x x 
University of Texas x x 
Other Work Based 
Council for the Assessment of x 
Experiential Learning (CAEL) 
ETS, “Fitness for Work” x 
Tennessee Board of Regents x 
Dual System, Germany x x x x x 
South Carolina x x x 
Photovoltaic Manufacturing x x x 
Consortium (with SUNY Poly) 
Note: “x” indicates the presence of a particular initiative characteristic, feature and/or point of leverage 
as indicated by the column header. Cells not marked with an “x” indicate the initiative feature either does 
not apply or fell outside the purview of our analysis. 
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Table 4c. Points of Leverage for Faculty Participation 


Directed 
funding 


Faculty time 


allocated to 
applied 


Recognition, 
reward 


Mandate 


Targeted to 
improve 
student 


learning 


learning 


SUNY Applied Learning Initiative 


x 


SUNY Works 
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System Level 
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United States 
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Washington 
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Sector Two-Year 
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Technical College System, GA 
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Sector Four-Year 
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Universities of Applied 
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Campus, Private Four-Year 
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California State Polytechnic 
Institute 


California State University, 
Monterey Bay 


South Dakota School of Mines 


Royal Melbourne Institute of 
Technology, Australia 


University of Michigan 


University of Texas 


Other Work Based 


Council for the Assessment of 
Experiential Learning (CAEL) 


ETS, “Fitness for Work” 


Tennessee Board of Regents 


Dual System, Germany 


x 


South Carolina 


x 


Photovoltaic Manufacturing 
Consortium (with SUNY Poly) 


x 


x 


as indicated by the column header. Cells not marked with an 
not apply or fell outside the purview of our analysis. 


Note: “x” indicates the presence of a particular initiative characteristic, feature and/or point of leverage 
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the fabric of the educational experience, and the University of 
Michigan’s University Research Opportunity Program (Urop), an 
initiative built from the ground-up with broad coverage. These 
five cases are illustrative, but not unique in every respect (we note 
similar approaches in other settings or campuses). 


1. System-Wide 


Higher education systems abroad have struggled with how to 
encourage work-based learning across institutions that may be 
very different. The three cases we examine illustrate the different 
system approaches to targeting and coordinating initiatives. Aus- 
tralia aims to encourage wider use of work-based learning, in 
part, by using funding and recognition to attract and support fac- 
ulty involvement in curriculum change. Germany’s Dual System, 
an apprenticeship approach, builds relationships with employers, 
encourages employer-education provider partnerships, and pro- 
vides a pathway to employment for students. The Universities of 
Applied Sciences in Switzerland, which provides practical train- 
ing and fully integrated experiences, shows how systems coordi- 
nate (at first weakly, then better with more funding) across 
different establishments. 


A. Higher Education Policy, Australia 

Australia’s higher education system is comprised of 
thity-seven public and two private full first-degree institutions 
that, although autonomous in operations and decision-making, 
come under national control for funding and regulation. Reforms 
dating to the late 1990s have led to consolidation in the sector, 
bringing smaller colleges of advanced education into a larger uni- 
tary system of universities.1° A separate segment of vocationally 
oriented institutions of Technical and Further Education (TAFE) 
provides for shorter post-high school studies. It is within this 
broad structure that complementary national initiatives have 
formed a base for system-level extension of work-based applied 
learning experiences. 

Interest in work-based applied learning grew out of evidence 
of uneven student performance — more than one-quarter of stu- 
dents failed to complete their studies — and concerns about the ad- 
equacy of a highly educated and well-prepared workforce.!” The 
recommendations of the government-appointed Bradley Review 
of Higher Education ushered in new conditions for extending 
“work integrated learning” (the term used in Australia) more 
widely to institutions and across study programs. The main rec- 
ommendations — to expand participation in higher education 
substantially, with additional resources allocated under a dra- 
matic shift to a demand-led funding system and performance 
monitored by a new standards and quality assurance agency (Ter- 
tiary Education Quality and Standards Agency) — continue to be 
rolled out.!8 For the purposes of this report, attention is given to 
recommendations relevant to work-based applied learning within 
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the broad reforms and to the means adopted within the system in 
response. 

The expansion of participation anticipated in the Bradley Re- 
view was understood to extend to students who previously 
would not have entered higher education, especially students of 
low socioeconomic status (SES), a target group of long-standing 
interest in government policy. Partly to address the needs of such 
new students as well as improve what had been weak perfor- 
mance to date, the Bradley Review stressed student engagement 
as an essential condition for learning. As part of a broader effort to 
gather information on teaching, learning, and performance, one of 
its forty-six recommendations stated: 


That the Australian Government require all accredited 
higher education providers to administer the Graduate 
Destination Survey, Course Experience Questionnaire, 
and the Australian Survey of Student Engagement from 
2009 onwards and report annually on findings.!° 


The recommendation has been partly advanced by the govern- 
ment. It is of particular relevance owing to the concepts of engage- 
ment incorporated in the Australasian Survey of Student 
Engagement (AUSSE). Modeled substantially on the National Sur- 
vey of Student Engagement (NSSE), which is used by a number of 
US colleges and universities, AUSSE includes an additional engage- 
ment scale on work integrated learning. Administered by the Austra- 
lian Council for Educational Research (ACER, similar to the 
Educational Testing Service), AUSSE was first used in 2007. 
Through 2012, more than 300 institutional administrations (some 
duplicates) of AUSSE have been carried out in Australia and New 
Zealand. As in the Bradley Review, findings with reference to 
work-integrated learning have been widely cited by higher educa- 
tion institutions and in public policy discussions.”° 

The Department for Education and Training has also made 
provision for targeted funding for quality improvements in teach- 
ing and learning in higher education. Through the Office for 
Learning and Teaching (and its predecessor), funds are allocated 
to support projects (with support for collaborative projects), 
secondments of faculty to the Department for Education and 
Training, and recognition of exemplary and innovative initiatives 
(and staff) in the institutions.”! 

Evidence of the wider impact of government strategies can be 
found in the activities of the nonprofit Australian Collaborative 
Education Network, (ACEN), the professional association of aca- 
demic and administrative staff, as well as representatives of em- 
ployers participating in work-integrated learning. Membership 
includes institutional, individual, and affiliate memberships. 
ACEN provides a venue for sharing experiences with work- 
integrated learning as well as approaches to implement and im- 
prove such experiences. Importantly, membership includes TAFE 
institutes as well as higher education establishments (and mem- 
bers of their teaching and administrative staffs), indirectly 
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supporting a growth point set out in the Bradley Review to foster 
new pathways to higher education. Through an annual confer- 
ence that brings these various parties together, work-integrated 
learning is given visibility as experiences are shared and made 
subject to discussion. 

ACEN itself has standing. It provides endorsement and indi- 
rect nonfinancial support for project proposals under consider- 
ation for government or third-party funding, under criteria that 
anticipate application for the wider membership (and beyond). In 
commentary on National Career Development Strategy Green Pa- 
per, for example, ACEN provided support for inclusion of work- 
integrated learning in all study programs, noting also that such in- 
clusion would require resources and incentives.” 


Key Features: 


m Campuses purposefully introduce well designed “work in- 
tegrated learning” into study programs; 


m “Work integrated learning” counts toward the degree. 
Points of Leverage: 


™ Gathering and monitoring information are important ways 
to draw attention to forms of applied learning (AUSSE); 


m Targeted funding for projects and recognition encourage 
and support faculty engagement in work-based applied 
learning (Office of Learning and Teaching, Department for 
Education and Training); 


m™ Teachers, staff, employers, and interested parties, orga- 
nized apart from government, have a venue to promote 
work-based learning by sharing experiences and collabo- 
rating (Australian Collaborative Education Network). 


B. Apprenticeship, Dual System (Germany)”* 

The “dual system” refers to a form of highly structured, 
strongly site-based apprenticeship offered in close partnership be- 
tween education providers and employers. In Germany (and 
some other continental European countries, most notably Switzer- 
land and Austria), entry into the dual system begins in upper sec- 
ondary education. About three-fifths of young people enter 
vocational education at this stage, and of them, nearly 
three-fourths opt for the dual system. Dual system preparation is 
offered for well over 300 trades. The dual system is of interest for 
program features and points of leverage that engage employers 
and support the delivery of the education and training in 
partnership. 

Apprentices participate in practical training for most of the 
week, with the contents, duration, and assessments set by guide- 
lines that have been developed and are reviewed by employers 
and are overseen by the Federal Ministry for Research (which has 
responsibility for vocational education). Alongside the practical 
training are classes provided in vocational and education training 
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schools (VET), of about twelve hours per week. Contents cover 
both general education and instruction specific to the trade, and 
come under guidelines established by the Conference of State 
Ministers of Education, the pre-eminent body overseeing 
education in Germany. 

More than is common for vocationally oriented education and 
training at this stage, the dual system provides for an extended 
period of training and experience under supervision in real work. 
Apprentices receive wages, beginning at less than half of what a 
skilled worker is paid to start. Over the three-to-four-year appren- 
ticeship, apprenticeship wages increase. The structured and antic- 
ipated engagement in work and the expenditures obliged under 
the system accompany strong involvement of employers. Beyond as- 
suming major responsibilities for the design of the apprentice- 
ships within fields (including curricula), employers supervise 
apprenticeships on site. Consortia formed around trades allow small 
companies to participate in training, and industry groups provide 
advice on guidelines. 

For students entering the dual system, the option is attractive. 
Costs are minimal, the education and training are well-structured and 
involves hands-on, actual work experience, and stipends are paid. The 
dual system is a path to a good job. Completion of the apprentice- 
ship leads to recognized qualifications that provide for employ- 
ment in skilled positions in any firm. While skilled workers are 
more likely to stay with the firms in which they have undertaken 
training, not all do. For example, Volkswagen’s Chattanooga 
plant, following the dual system approach in cooperation with the 
Tennessee Technology Center at Chattanooga State Community 
College, has found that trained workers are more likely to leave 
for employment elsewhere than is the case in Germany. 

Expenditures are shared. State and local authorities pay for the 
instruction provided in VET schools, while training firms bear the 
costs of on-site training. The larger share of expenses is borne by 
firms in the industries concerned, even taking into account the 
contribution of apprentices to production. However, some of the 
firm’s expenses for the dual system might be incurred through 
company-based training, if the dual system did not exist. Firms 
generally have been satisfied with the quality of the workforce, and 
have had a hand in developing an understanding in apprentices of 
the culture of work in the company. Further, firms use the dual sys- 
tem as means of recruitment, and so avoid costs in that area both 
for hiring and for replacements of recruits who are ill suited. 

The dual system has encountered some difficulties, as compa- 
nies are less willing to take on apprentices. Some attribute compa- 
nies’ reluctance to very tight regulations, costs, and education and 
training that is insufficiently broad enough to enable skilled work- 
ers to adapt to evolving needs in the industry. 


Key Features: 


m Work experiences are well structured, allowing for cumu- 
lative learning; 
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™ Students participate in ongoing workshops and seminars, 
alongside work experiences. 


Points of Leverage: 


@ The dual system has very extensive employer engagement 
and investment, with arrangements for small- and 
medium-sized enterprises; 


m@ Itis a true partnership among ministries, employers, and 
education providers; 


M™ Itis attractive to students because it offers structured 
learning, wages, and a clear pathway to employment; 


m However, it is a delicate balancing act. The problem is not 
so much the cost, but the profile of knowledge, skills and 
attitude. If apprentices are too narrowly trained, employ- 
ers will invest less. 


C. Universities of Applied Sciences, Switzerland 

In Switzerland, as in several continental European countries 
(Netherlands and Germany, for example), degree studies beyond 
secondary education are provided in two types of higher educa- 
tion — universities (to which Swiss Federal Institutes of Technol- 
ogy may be included) and universities of applied sciences (UAS). 
The UAS came out of a 1990s reform to consolidate, strengthen, 
and upgrade mostly applied technical and professional programs 
to accommodate growing demand. The UAS regard themselves as 
“equivalent but different” than universities. The fields covered in 
the UAS study programs initially came under national legislation 
(excepting teacher education). UAS students may come from any 
prior educational stream (under set admissions guidelines), al- 
though a smaller share of students entering UAS programs have 
followed academic paths through upper secondary school. Provi- 
sions are in place for students who have followed a vocational 
path through and beyond upper secondary education to qualify 
for entry into either universities or UAS, and a growing number of 
students coming out of the vocational streams are pursuing both 
routes.*4 About one-third of students in higher education at this 
level are enrolled in UAS. The Swiss experience under UAS fea- 
tures the use of practical training and experience fully integrated 
within the study programs. 

In the UAS, students follow well-structured study programs that 
combine academic coursework with workshops in individual 
courses. For a bachelor’s program in conservation and restoration, 
for example, students may participate in regular course modules 
for the first three days of the week and workshops with instruc- 
tion in application from professionals for the remainder of the 
week. During a seventeen-week term, students may spend six 
weeks in an external practice module, supervised and evaluated 
by a qualified professional. Two such six-week blocks may be 
combined into an internship.” 
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Through the UAS reform, more than seventy-five specialized 
institutions were merged into seven regionally based UAS (five in 
German-speaking regions, one in French-speaking regions, and 
one in the Italian-speaking canton).7° The Swiss experience is also 
of interest in how initiatives with work-based experiences are co- 
ordinated across different establishments. Although coming un- 
der a single system, each UAS operates differently. Moreover, 
early experiences showed differences in the efforts made in the 
early years to establish more co-ordination among the initiatives 
brought under the oversight of each UAS. Indeed, those UAS that 
introduced weak co-ordination and centralized management 
aimed for less ambitious reforms.?” With funding and other sup- 
ports, co-ordination has increased such that study programs may 
be offered jointly, affording students work experiences aligned 
more closely with their interests. In the case of the UAS bachelor’s 
program in conservation and restoration, for example, four UAS 
cooperate, offering the same “core” coursework but affording stu- 
dents work experiences that vary by location. 


Key Features: 


m The UAS has full integration, preserving a feature of ad- 
vanced level vocational studies as the UAS were estab- 
lished at a level equivalent to universities. 


Points of Leverage: 


™ The UAS is able to coordinate among establishments un- 
der directives and funding, some from the national level. 


2. Campus-Wide 


Individual campuses, too, have tried to extend work-based 
learning opportunities to all students. We provide two cases. 
Northeastern, which has a strong, well-supported cooperative 
learning experience, is known for the integration of work and ed- 
ucation. This experience, however, is expensive to replicate. The 
University of Michigan’s Undergraduate Research Opportunity 
Program is a ground-up faculty initiative to provide research op- 
portunities to undergraduates that has been scaled up. It operates 
with a more modest infrastructure than Northeastern’s, and bene- 
fits from strong faculty and student buy-in (otherwise referred to 
as a “ win-win”). 


A. Northeastern University Co-ops 


Northeastern offers study programs oriented toward employ- 
ment. Among its various applied learning experiences, Northeast- 
ern is recognized for co-op learning. The “co-op experience” 
includes as many as three six-month periods of structured assign- 
ments with an employer in the course of a bachelor’s or master’s 
degree program. Although students are not required to undertake 
the co-op experience, many do: in 2013-14, 9,823 students were en- 
gaged in such experiences, against a total undergraduate enroll- 
ment of 17,101 (as of fall 2013). Although a large proportion of 
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students in the co-op experience are in their second or third struc- 
tured assignment, some 92 percent of students participate in at 
least one co-op experience and 75 percent do two or more. With 
respect to its reach across all programs, co-op learning at North- 
eastern is an illustrative example of scaled up work-based applied 
learning. 

Northeastern’s co-op experience utilizes key initiative features 
of work-based applied learning. 

Co-op experiences are well-structured and fully integrated in the 
curriculum. Learning outcomes associated with experiential learn- 
ing (i.e., service learning, applied research activities, internships, 
and co-op learning) are clearly set out, as are templates of the 
study program with possible sequencing of coursework and co-op 
experiences. Faculty engagement is extensive and substantive, in 
the light of the obligation to devise, implement, and refine inte- 
grated study programs and the very large share of students who 
participate in co-op learning. 

The first co-op experience begins no earlier than the second semester 
of the sophomore year. Students may participate in two sixth-month 
co-op experiences (in a four-year program) or three co-op experi- 
ences (for a five-year program). Each student returning from a 
co-op experience is expected to produce a reflective account of the 
experience and, in particular, how the experience related to what 
has been learned in courses. The program allows for different 
ways to develop that account, e.g., “participating in company 
seminars, faculty conferences, one-on-one meetings with the 
co-op coordinator, writing assignments/or presentations.” 78 

Support for students on co-op is provided through the aca- 
demic advisor and a co-op coordinator, both located in the stu- 
dent’s college. In most colleges, a co-op coordinator teaches a required 
one-credit course as preparation for the co-op experience, and works 
with students to help them develop and use job search strategies 
and techniques, such as securing job referrals and/or references 
from their own networks, attending job fairs, participating in stu- 
dent information sessions on opportunities, participating in mock 
interviews, and undertaking research on companies in the field of 
interest. More than fifty co-op coordinators support students and 
the academic programs in which they are enrolled, with some col- 
leges having more than one. 

The University reports placements with 2,200 employers on 
six continents.”? For employers, co-op “student-employees” join 
the staff and work alongside regular employees. Like the job mar- 
ket more generally, employers make the decisions on hiring. Those 
hired take up structured assignments, and are paid a wage (but 
not company benefits, e.g., health insurance). The period of the 
co-op experience, at six months, is long enough to permit the 
co-op student-employee to “get real work done.” At the same 
time, employers can evaluate co-op student employees for poten- 
tial recruitment. The university works with employers, both sup- 
porting the process of placement — and eventual potential 
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recruitment — and involving employers in activities to support 
co-op arrangements and the study programs in the fields con- 
cerned. An employer handbook lays out clearly what is provided 
by the university (and so not needed by the employer), as well the 
responsibilities of all parties. The aim is sustained engagement. 

Employers are welcomed to campus for a variety of activities 
including student workshops, career fairs, and meetings with 
staff. In addition, “employers in residence” are provided with an 
office for single or recurring visits to meet with students regard - 
ing career development. 

The co-op experience is credited with very strong employment 
outcomes. The university reports 90 percent of graduates are em- 
ployed within nine months of graduation, half are employed with 
their co-op employer, and 85 percent are working in their field of 
study.°° For Northeastern, the profile of graduates is strongly ori- 
ented toward the professions. For 2012-13, 20.49 percent of bache- 
lor’s degrees were conferred in business; 12.5 percent in health 
professions; 12.47 percent in engineering, 11.07 percent in con- 
struction trades, and 7.6 percent in communications/journalism. 
By comparison, social sciences, biology and psychology ac- 
counted, respectively, for 11.7, 7.3, and 5.52 percent of bachelor’s 
graduates in that year. As indicated, students in all fields 
participate in co-op experiences. 

Takeaways: Northeastern’s co-ops offer instructive lessons for a 
scaled up work-based learning initiative on a campus. 


Key Features: 


™ The emphasis on integrated curriculum means that work 
and learning are not two-separate spheres. Students think 
about how learning applies to work and vice versa; 


m Applied learning experiences are undertaken throughout 
the study program, allowing for cumulative learning. 


Points of Leverage: 


™ Co-op and applied learning figure strongly in the mission 
and work of the campus as a whole; 


™ University resources are directed to support the 
work-based applied learning experiences; 


m™ Academic and work-experience supports are both located 
within each of the colleges, reducing the barriers for stu- 
dents and employers to participate; 


m The co-op experience offers sustained engagement with 
employers, which gives students a richer and more mean- 
ingful experience at the same time that employers are 
more willing to invest in the program and individuals in it. 
It has clear expectations, laid out in an employer hand- 
book, and a dedicated staff to foster and sustain the 
engagements; 


m Employers pay wages, but do not provide benefits. Em- 
ployers have student-employees for at least six months 
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and can bring back them back for another six months if it 
works, essentially acting as prerecruitment. 


B. University of Michigan: Undergraduate Research 

Opportunity Program (Urop) 

Michigan has offered undergraduates the opportunity to par- 
ticipate in faculty-mentored research since 1989. The program 
originated with faculty in a single department who wanted to en- 
courage and engage under-represented and less prepared stu- 
dents, with the aim of improving retention. Although 
under-represented students remain a target population (see be- 
low), eligibility was broadened to include all students and cam- 
pus funds support the program. Currently, students from all 
colleges participate in applied research opportunities and Urop 
posts 1,300 job opportunities. Urop is an example of a scaled-up 
applied learning initiative at the campus level. Michigan’s Urop 
program has ambitions and specificities broadly similar to other 
campus-wide initiatives elsewhere, such as Brown Connect and 
Engaged Cornell. 

Urop utilizes key features of applied learning and benefits 
from several points of leverage to support and sustain the cam- 
pus-wide initiative. 

Those associated with Urop say the applied research experi- 
ences are attractive to both faculty and students: It is well-structured 
to meet the needs of both.5! Although submitted on a standard for- 
mat, the applied research experiences offered by faculty follow no 
set model as long as activities are appropriately defined and situ- 
ated within the wider research project of the faculty member. The 
only expectation is that students have a substantive learning expe- 
rience. In this way, the student experience is well-structured and, 
at the same time, aligned with methods and nature of the faculty 
research project. 

A key component of Urop is a set of workshops that develop basic 
research skills. The workshops partly aim to build and reinforce 
dispositions for structured research activities. They also introduce 
basic understandings about the process of inquiry/research and 
how one goes about gathering information and working with it. 
Urop gradually has come to rely on students with experience in 
Urop to “tutor” those entering their first applied research experi- 
ence. There is a fair amount of peer involvement, even peer con- 
trol. While not strictly speaking an applied research experience, 
peer teaching/mentoring requires understanding of what is being 
taught (and learned) and organizing sessions to convey informa- 
tion and receive feedback. Faculty members appreciate the work- 
shops because the students who work on their projects come with 
basic knowledge, dispositions, and skills. In that sense, the de- 
mands on faculty are reduced. For Urop, the workshops have 
very modest resource requirements. 

Students interview for Urop positions. The interviews are taken 
seriously by both sides, and neither the faculty member nor the 
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student is assured of a “first choice” in the process. Like job inter- 
views more generally, interviewing for this applied learning initia- 
tive means it is possible that a position will go unfilled or a student 
will not be placed. Urop staff oversees the process, thereby facilitat- 
ing the matching and helping both students and faculty. 

Students apply for Urop positions early in their studies. Students 
may continue in Urop or other research experiences. Some stu- 
dents continue on with Honors thesis projects while others may 
participate in faculty research projects outside of Urop. Faculty 
members have come to value students who begin Urop early, ap- 
preciating that their contributions to faculty research increase 
with experience. Less training is needed, and for the student, 
there is greater potential for recognition in publications (some 
with secondary authorship) or presentations. Doctoral candidates 
and post-docs also work with Urop students, gaining experience 
in return for more intensive one-on-one supervision. 

In most instances, participation in Urop is credit bearing (see be- 
low). Assessments of student performance are based on assignments that 
vary among projects (from writing an extensive literature review 
or running an experiment to administering a survey and making a 
presentation). Urop “alumni” value both the research experience 
itself and the range of activities and supports provided through 
Urop, with under-represented students more likely to value each 
of these aspects than other students.*? 

Urop has a small staff, drawing financial support from the col- 
lege in which it is formally located (the largest on campus). The 
program successfully attracted donor support to cover student sti- 
pends for a summer program. Some Urop students receive 
work-study support as well. For these students, Urop assignments 
may be viewed as more interesting and meaningful than a routine 
assignment to an office or unit on campus. Work-study students 
do not receive academic credit for the Urop experience, but they 
do register for one credit for the workshops and academic super- 
vision of their work. Urop staff handle all of the paperwork, re- 
lieving the faculty of the responsibility for reporting requirements. 

Most Urop projects are located on-campus, but there has been 
some experience with work-based and service-based learning un- 
der the project. Urop has provided community-based research ex- 
periences for students working with community organizations 
that come under a broader university engagement effort in De- 
troit. The research experiences are supervised by university staff, 
not staff of the community organization. 

Under-represented students and those with weak preparation 
continue to participate in, and benefit from, Urop. Beyond ar- 
rangements for work-study students (who receive support based 
on financial need) discussed above, Urop attracts community col- 
lege students to a summer program, increasing the flow of such 
students to the university and helping with their academic transi- 
tion. This is a stand-alone program to raise the share of talented 
low income/under-represented students into leading colleges and 
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universities who otherwise might not apply.** This Urop-con- 
nected project has been successful: 85 percent of students in the 
program have enrolled at the university, though there is no alter- 
native with which to compare them. 

Urop has been the subject of rigorous and varied evaluation to 
estimate the effects of participation in the research experience. The 
findings, focusing primarily on under-represented students and those 
with weak preparation, show that participation in Urop is associated 
with higher rates of retention, a greater likelihood of entering post-gradu- 
ate study, and behaviors judged to be more proactive, when compared to 
an appropriate control group.*4 

Takeways: There are many unique elements of the Urop pro- 
gram. But particular design features make the program successful 
and are important takeaways for scaling-up initiatives at SUNY. 


Key Features: 


m Urop learning experiences are broadly structured and rec- 
ognized for credit; 


™ Urop offers opportunities aligned with student interests, 
with projects in every college. 


Points of Leverage: 


@ Urop isa “win-win” program: students get credit for par- 
ticipation, they engage in a new form of learning, and 
some are able to be paid through work study while faculty 
have support for their research projects and do not have 
extra effort for administrative matters. A similar approach 
may be effective when applied to employers (rather than 
faculty); 


m Like employers in the dual-system or co-op placements at 
Northeastern, the timeframe —which starts early and ex- 
tends over a number of years—is an attractive opportunity 
both for students to grow and learn in the position and for 
their faculty supervisors to have better trained and more 
expert student assistance as the years progress; 


™ Peers play an important role in the Urop program, which 
suggests important extensions for other types of applied 
learning initiatives. More generally, applied learning ini- 
tiatives might benefit from working with student organi- 
zations and campus departments to raise the visibility and 
relevance of opportunities; 


™ Urop was able to generate resources for summer support 
outside the university, through the generosity of donors. A 
similar principle may be accomplished in work-based ap- 
plied learning, for example, through named internships 
with visibility across departments. 


C. How SUNY Addresses Key Design Features 


Although these five cases provide important illustrations of 
seemingly successful or very recent applied learning initiatives, 
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SUNY initiatives already in place at some campuses manifest 
some key design features. 

Stony Brook has launched a co-op initiative that is well struc- 
tured with clear expectations for students and stakeholders. Stony 
Brook devised specific evaluation requirements “to avoid poten- 
tial problems early in the process and to make the initiative suc- 
cessful.”5 They are as follows: 


m™ Students should be continually informed by the employer 
of what is expected of them and how they perform; 


m™ Site supervisors must communicate regularly with stu- 
dents, know when work is done well, and give the oppor- 
tunity to improve/correct problems before the end of the 
assignment; 


® All co-op students will evaluate the program internally, in- 
cluding evaluation of supervision. This enables them to 
identify strengths and potential problem areas in the 
program; 

™ Mid-semester and final evaluations must be completed; 


m™ Learning objectives are set by students with supervisor’s 
guidance; 


mM ePortfolio became a mandatory piece of the process to as- 
sist students with better articulation of skills and qualifica- 
tions learned through experience. 


Likewise, several schools have updated and revised their in- 
ternship manuals to provide clear guidelines as to the structure of 
initiatives. Monroe Community College, for instance, “updated 
the employer guide for co-ops and internships making it more 
professional in appearance” ** while SUNY Orange created sepa- 
rate internship manuals for students and employers. The latter 
now “includes a section on paid vs. unpaid internships and ex- 
plains the Department of Labor (DOL) regulations governing un- 
paid internships.” °” 

Stony Brook, Fashion Institute of Technology (FIT), Broome 
County Community College and Buffalo State include organized 
workshops or seminars on campus to orient students toward their spe- 
cific fields of study. Stony Brook developed introductory sessions 
while scheduling mandatory orientations and co-op information 
sessions for students.** These sessions were deemed important by 
the Career Center because co-op placements had not previously 
been available on campus. Noting their success, representatives 
from the Career Center indicated that, “students started paying 
attention when we initiated information sessions. Students were 
coming to hear about these opportunities.... We would love to ex- 
pand the program but we just don’t have the resources.” 

At FIT, all students seeking an internship must complete one se- 
mester of required prep prior to placement. According to Andrew 
Cronan, director of FIT’s Career and Internship Program, “the prep 
process is designed to make sure students get professional training 
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so they represent us and themselves well.” Through their involve- 
ment with SUNY Works, FIT developed a new set of learning out- 
comes for the required prep process so that students who complete 
multiple internships can test out of the requirement. 

Broome Community College sponsored job readiness activities 
for students that included weekly, campus-wide student work- 
shops led by local employers and placement specialists. Topics 
covered included careers in demand, resume writing, interview- 
ing, effective networking, and job searching. To supplement these 
efforts, various disciplines conducted mock interview days 
throughout the 2014 spring semester in which employers inter- 
viewed students in twenty-minute intervals and completed evalu- 
ation sheets. Students were then presented with constructive 
feedback from these evaluations.*! Through collaboration with 
Geneseo, Broom has since identified “the need to develop a Pro- 
fessional Skills Preparatory Course.” 

Buffalo State designed and is piloting a new Professional Skills 
Development Module (PSDM). The module consists of a hybrid 
curriculum, “comprised of four tracks with the overall goal to en- 
hance the quality of the internship experience by equipping stu- 
dents with professional skills before they begin their internship 
placement. Eight faculty members, from a variety of majors [i.e., 
psychology, biology, sociology, theater, music, business, fashion 
and social work], were selected to develop and pilot the modules 
in their internship classes. Each pilot class will select the work- 
shops/activities/online modules to be completed from among the 
tracks.”4? Buffalo has also developed a pre- and postsurvey to as- 
sess overall student satisfaction and confidence level. 

Even though there are benefits from completing internships 
early, most SUNY students who complete internships do so later 
in their college career. SUNY may pursue the possibility of foster- 
ing opportunities for applied learning experiences earlier in the degree 
program. Andrew Freeman, director of academic services at Mon- 
roe Community College (MCC), summed it up this way: “When 
you talk about career services and planning upfront, what I was 
able to uncover was this idea that there’s a lot that MCC offers in 
a variety of offices but we haven’t done a good job giving a stu- 
dent a laundry list in inspiring them to pursue these opportuni- 
ties. It’s just been if a student is able to stumble in the right office 
maybe they get the right information. Collecting what we do, see- 
ing how it flows, and then being able to encourage students in a 
logical order will bring clarity to things at MCC.” Monroe has 
since developed a Career Action Plan that lists activities a student 
should engage in to maximize their employability after gradua- 
tion including internships. 

Albany has reported a similar experience. Comments made by 
the Interim Associate Director of Career Service Noah Simon indi- 
cate that prior to implementation of SUNY Works, the Office of 
Career Services was unaware that many students were complet- 
ing experiential opportunities in the School of Education 
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(primarily in the domain of field work) as well as in the Study 
Abroad program.*4 

Unfortunately, many students remain unaware of opportuni- 
ties that exist despite campus efforts to reach students through 
email marketing campaigns, online postings, informational flyers, 
and promotional videos. For example, Fulton-Montgomery Com- 
munity College developed two television commercials detailing 
opportunities for experiential learning on campus. According to a 
project report submitted to SUNY Central in October 2014, “the 
marketing and outreach materials ... allowed our campus to tar- 
get local residents interested in degree programs that incorporate 
experiential ‘real world’ learning. Since unemployment rates in 
the area our campus serves are among the highest in the state, this 
opportunity to connect residents to programs designed to get 
them quickly into the workforce was sorely needed and very well 
received.” 

Other campuses, including Onondaga Community College, 
Stony Brook, and the University at Albany, rely heavily on faculty 
to encourage students to participate in internships. According to 
one representative from Career Services at SUNY Albany, “SUNY 
Works has provided us with a way to engage faculty and other staff 
in the increasing value of internships and applied learning experi- 
ences. Through these conversations we have been able to speak in 
more classrooms and many faculty and staff have increased conver- 
sations and initiatives with students and employers about intern- 
ship opportunities. These relationships led to an increase of over 17 
percent in employers at our annual career fair and an increase of 
over 25 percent in student attendance at this event.” 46 

Broome Community College has developed a good model. 
Specifically, freshman student orientation sessions feature applied 
learning presentations by the Career Readiness & Job Placement 
team in collaboration with the Service Learning and Civic Engage- 
ment Coordinators. An information table is also accessible 
throughout orientation sessions.‘” 

Marketing campaigns have also been utilized at places like 
Hudson Valley Community College, Monroe, SUNY Adirondack, 
and SUNY Orange to develop new relationships with community 
partners and employers to inform students about the value and availabil- 
ity of co-ops and internships. 

Faculty and staff throughout SUNY have expressed concern 
over the potential for work-based experiences to interfere with de- 
gree completion. According to Statewide Co-Op Curriculum Co- 
ordinator Bill Ziegler, the question is, “How do you add six 
months work to that [the curriculum] and still get it done in four 
years or less?”4° Campuses have responded in a number of ways 
to promote sustained learning opportunities that are integrated through 
academic curriculum and help build cumulative knowledge over time. At 
Stony Brook, for instance, faculty and staff are experimenting with 
a parallel co-op initiative that differs from Northeastern in that 
students work part-time while being enrolled full-time. This 


Rockefeller Institute Page 29 www.rockinst.org 


Higher Education Applied Work-Based Learning at the State University of New York 


approach was taken in large part because faculty were “not sup- 
portive of the idea of students leaving and working full time for a 
semester — it breaks the students’ academic sequence and can 
postpone graduation.” *? Stony Brook has also engaged faculty 
and administrators to discuss incorporating the co-op initiative 
into the curriculum of peer education programs.°*? 

Ziegler has indicated that curriculum development requires 
faculty and staff to “think outside the box” when creating intern- 
ships. “We tried to follow that model where even if you’re an art 
history major there’s an internship for you.... For example, I have 
an art gallery. We found public space to show student work and I 
have a student who is the curator.”5! Not surprisingly, some ma- 
jors prove more difficult to develop internships than others. At 
Onondaga Community College, for example, Internship Coordi- 
nator Rose Martens expressed some difficulty developing intern- 
ship courses in the humanities.5? By comparison, Onondaga 
established three new courses in the Interior Design program to 
give students more flexibility when seeking internship credit. Stu- 
dents may choose from courses that are one, two and three credits 
and require sixty, 120, and 108 hours, respectively.*? New intern- 
ships were also established for the new Nuclear Technology 
program. 

While every SUNY Works pilot campus offers credit-bearing intern- 
ships, the number of students completing noncredit bearing internships 
is unknown. That being said, some schools are adopting new poli- 
cies designed to designate courses with an experiential compo- 
nent. For example, Cayuga Community College developed an EL 
Designation Policy which allows faculty to designate their courses 
as EL on course descriptions and transcripts.*4 This policy took 
two years to put in place and required feedback and buy-in from 
the faculty association. 

Finally, successful initiatives offer alternative assessment. Although 
this occurs within SUNY, it does not appear to be very common. FIT re- 
cently developed a Transfer of Credit Policy for academic intern- 
ships. This policy allows students who complete an academically 
equivalent internship within or outside of SUNY to transfer those 
credits to FIT. Despite the relatively low-cost of implementation, 
FIT anticipates “this will not be a highly used option for students” 
because FIT’s “internship program is valued and desirable for stu- 
deénits.“5 

SUNY also has an expert in prior learning assessment. Nan 
Travers currently serves as the director of the Office of 
Collegewide Academic Review at Empire State College. Her work 
focuses mainly, “on the policies and practices of self-designed stu- 
dent degree programs and the assessment of prior college-level 
learning.”°° As founding co-editor of PLA Inside Out: An Interna- 
tional Journal on the Theory, Research, and Practice in Prior Learning 
Assessment, Travers’ knowledge and expertise lend themselves to 
future growth in this area. Nonetheless, widespread implementa- 
tion of PLA’s remains uncommon throughout most of SUNY. 
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D. Points of Leverage 

Universities and colleges perceive a great benefit in providing 
applied work-based experiences for their students. As Michael 
True, Washington Internship Institute board member, explains: 


Colleges helping students find internships is a growing 
trend. Schools raising money to fund unpaid internships 
is a growing trend. Use of alumni in this way is a grow- 
ing trend. All of this is on the upswing because it all 
makes it that much easier for students to transition to the 
workplace....°7 


If applied learning offers benefits, how can opportunities be 
expanded or brought to scale? The answer, we suggest, can be 
found in key points of leverage to promote and expand applied 
learning opportunities. Points of leverage are ways to extend 
good or effective applied learning experiences, at system or cam- 
pus levels. Although there is scant research testing these points of 
leverage, we draw on expert advice and examples of relevant ap- 
proaches adopted at system or campus levels (beyond the five il- 
lustrative cases outlined above). We group the points of leverage 
under three major types: requirements, incentives, and buy-in. Sit- 
uating points of leverage advanced by other systems and univer- 
sities against policies, strategies, and experiences in SUNY Works, 
shows potential areas for policy development in SUNY Works. 


1. Requirements 

Systems have expanded applied learning initiatives by man- 
dating or requiring participation of students and faculty. The Uni- 
versity System of Maryland requires students to earn credits 
outside the classroom, including credits earned in internships. 
The Youth Guarantee in Finland goes much farther in supporting 
opportunities for work, including a guarantee that anyone under 
twenty-five will be offered employment if they are job seekers. 
Massachusetts’ new “Civic Learning” policy is a directed, if 
broadly based, mandate that obliges all students in the system to 
acquire “applied competencies that citizens need” without refer- 
ence to how students should acquire those competencies.*® 


M University System of Maryland. Within a comprehensive Ef- 
fectiveness and Efficiency (E&E) initiative rolled out in 
2003-04, the University System of Maryland introduced a 
requirement that students must earn at least 10 percent of 
the 120 credits for the bachelor’s degree outside the class- 
room. The requirement can be met through online courses, 
study-abroad programs, or Advanced Placement tests, but 
also through internships.*%? 


@ Youth Guarantee, Finland. In setting out a new direction for 
policy, the Ministry of Education has advanced the view 
that “young people have the right to construct and the re- 
sponsibility for constructing their own future.” That new 
direction has been manifested in policies that oblige the in- 
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volvement and engagement of young people through sup- 
port and recognition of nonformal education, flexible 
pathways, and apprenticeships. Introduced in 2013, the 
Youth Guarantee ensures that anyone under age 
twenty-five (and graduates under age thirty) will be of- 
fered employment, trial employment, or tailored study if 
registered as a job seeker. More generally, about half of 
students in Finland report being employed while study- 
ineO! 


2. Incentives 

Rather than require student or faculty participation, systems 
can encourage it, most often through increased funding. Trade 
Adjustment Assistance Community College and Career Training 
(TAACCCT) grants, for example, encourage partnerships between 
campuses and employers by providing funds. Similar initiatives 
can be found, with state funds, within the Technical College Sys- 
tem of Georgia and, with respect to targeted and grounded re- 
search, at the University of Texas. 

Individual campuses have also incentivized greater participa- 
tion, again through increased funding opportunities. Brown Uni- 
versity launched BrownConnect, which develops and lists 
internships and other applied learning experiences. It also pro- 
vides stipends for students who participate. Similarly, Cornell 
launched its own initiative, Engaged Cornell, to encourage com- 
munity engagement. Cornell anticipates $150 million over ten 
years to support community partnerships, curriculum change, 
and faculty development. 


@ Trade Adjustment Assistance Community College and Career 
Training (TAACCCT) Grants. Funding under TAACCCT is an 
example of external funding to bring about campus- 
employer partnerships for skill training. The most recent 
round of TAACCCT grants focus on regional employer- 
community college partnerships that engage small and me- 
dium enterprises (SMEs) as well as large employers.°? 


m BrownConnect. Launched in November 2014, 
BrownConnect serves as infrastructure for Brown students 
in summer internships as well as other applied experi- 
ences. The opportunities are available to all students, re- 
gardless of financial need. Jointly provided by the careers 
and alumni offices and supported by campus resources, 
BrownConnect develops and lists internship and applied 
learning opportunities and generates and allocates fund- 
ing. During a one-year pilot phase, 154 new internships 
were established. Through BrownConnect, students re- 
ceive financial support if they undertake appropriate in- 
ternships that offer no or low stipends. In the pilot year, 
some 254 students received such support. Funding comes 
through donors, department resources, and financial aid. 
While the internships or other forms of applied learning 
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are expected to provide appropriate learning experiences, 
the experiences appear to complement academic study. 


m Engaged Cornell. In October 2014, Cornell introduced En- 
gaged Cornell, an initiative to promote the participation of 
all students in “high-quality” community engagement. 
Participation is voluntary. The initiative was launched 
with a $50 million donation from the Einhorn Family 
Charitable Trust, toward $150 million in funding over ten 
years. Funds are intended to support a substantial growth 
in community partnerships, curriculum change in all fields 
and departments with aligned learning outcomes, and tar- 
geted faculty development. Engaged Cornell potentially 
will become a distinctive feature in the university’ profile, 
attractive to potential students and faculty. The initiative is 
housed in the office of the vice provost for land-grant af- 
fairs.o4 


3. Buy-In 


Systems may also encourage greater participation in 
work-based applied learning through federal and state policy de- 
velopment. Indiana and Illinois have created new work-study 
funds to support internships, including in the private sector. Sys- 
tems have worked with state and regional governing bodies to 
promote funding or tax concessions for employers who make a 
commitment to create or promote jobs — a strategy that might be 
purposefully extended to include internships. 

Campuses, likewise, encourage buy-in from employers. Expe- 
rience with strategies and approaches elsewhere suggest that em- 
ployer engagement is facilitated through arrangements that 
reduce administrative and other demands while affording bene- 
fits. Beyond those already identified in the illustrative cases, tar- 
geted approaches include working with employer consortia for 
workforce training, for example, the Photovoltaic Manufacturing 
Consortium. 

Campuses can encourage faculty buy-in by growing and pro- 
moting internal initiatives, as in the University of Michigan’s 
Urop program (discussed above). Campus initiatives may encour- 
age students to take internships and to complete work-based pro- 
jects as part of their curriculum, like California State Polytechnic 
Institute (Cal Poly). Strong work-based initiatives may encourage 
students to participate simply because they get results. The South 
Dakota School of Mines, for example, has well-paid work-based 
opportunities with many different employers. Work is integrated 
as a value in the university. Drake and Duke, in less directed 
ways, provide opportunities that attract students to applied learn- 
ing (including work-based applied learning).© 

Over time, work can become a core part of the education at a 
particular campus. These schools are able to get faculty and stu- 
dent buy-in through recruitment: individuals may choose them 
because of their work-based emphasis. Northeastern, described 
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above, is known for its co-op experience. Liberal arts schools, like 
Berry College and Berea College, integrate work fully into their 
curriculum and educational experience. California State Univer- 
sity, Monterey Bay (CSUMB), provides support for service learn- 
ing, a requirement for all students integral to the CSUMB 
mission. The Royal Melbourne Institute of Technology (RMIT) 
has emphasized work since its origins. Unlike Northeastern, 
Berry, or Berea, the RMIT incorporates technical programs as well 
as traditional bachelor’s, master’s, and doctorates. 


System-level: 


B Work Study in Illinois, Indiana, and Washington. Work study 
is a key growth point in state and federal policy develop- 
ment. In Illinois and Indiana, state funds have been set aside 
to provide work study support for students undertaking 
internships to foster skills and knowledge for employ- 
ment. In Washington, the initiative is state-funded and not 
restricted to private sector involvement. In the Federal 
Work-Study program, matching requirements and limita- 
tions on the share of funds that can be allocated to pri- 
vate-sector placements make these placements less 
attractive. However, proposals to facilitate the use of Fed- 
eral Work-Study funds for private sector placements, and 
to focus work study placements much more on relevant 
work-based learning experiences, have been advanced for 
consideration in the re-authorization of the Higher Educa- 
tion Act as well as in states.®” A further feature of interest: 
Under the Job Location and Development Program (JLD) 
in Federal Work Study, colleges and universities can sup- 
port the development and visibility of off-campus job op- 
portunities for all students, particularly those providing 
work-based applied learning experiences. 


= Commitments in economic development grants or conces- 
sions. State and regional funding or tax concessions may 
include terms for commitments for jobs.°* Where job com- 
mitments are expected or regarded favorably, terms might 
be purposefully extended to include as relevant and ap- 
propriate structured internships. Such an approach aligns 
with and expands such supply side (if job-driven) ap- 
proaches as TAACCCT (above). 

® Photovoltaic Manufacturing Consortium. Consortia used 
in workforce training, identified in some regions as “sec- 
tor-based” (e.g., hospitals), permit enterprises of various 
sizes, capacities, and levels of recruitment to participate in 
the development and delivery of work-based applied 
learning experiences. The Photovoltaic Manufacturing 
Consortium is one example through which workforce 
training at all levels (including internships) is provided in 
partnership with SUNY Polytechnic and a range of public 
and private entities, with private and public funding. The 
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approach might be applied (with or without funding) to 
engage groups of employers to work with colleges to de- 
velop work-based applied learning experiences for a wider 
range of students and purposes, as has been the case in 
North Dakota and Connecticut. Chambers of Commerce or 
economic development or workforce development entities 
might serve as the appropriate convening organization. 


Campus-level 


@ Learn by Doing at California State Polytechnic Institute. Cal 
Poly provides degree programs in mostly professional and 
technical fields, and emphasizes in its academic approach 
“learn by doing.” One overarching manifestation of this 
applied orientation is a required senior capstone project in 
all fields. The project, in the form of a design or research 
study, presentation, or performance, can draw upon an in- 
ternship or co-op experience (expected in some fields) as 
well as related to any other work or experience. Senior 
projects have visibility, with some noted as the origins of 
start-ups. The most recent survey of graduates, for 
2012-13, shows that 75 percent were employed (69 percent 
full-time, 6 percent part-time) and 13 percent were in grad- 
uate school. Of those employed, 92 percent report jobs di- 
rectly or somewhat related to the field of study.” 


@ South Dakota School of Mines. In this long-standing initia- 
tive, all South Dakota School of Mines’ students are en- 
couraged to have relevant work or research experience 
prior to graduation. More than three-fourths of graduates 
report such experiences. Co-ops are often six-eight months 
(semester + summer). In 2013, students interned with more 
than 165 employers in thirty-four states, Canada, and Ger- 
many. Many work-based experiences are paid: the average 
co-op/intern salary for 2012-13 was $17.03/hour, not in- 
cluding housing allowances, bonuses, and relocation ex- 
penses for work experiences taken away from campus. 
The emphasis on applied learning carries through to a 
one-year project, required in the senior year. 


m Work Experience Program at Berry College. At Berry College, 
a small private liberal arts college east of Atlanta drawing 
students mostly from Georgia and neighboring states, 
more than 90 percent of students participate in the Work 
Experience Program. The program comes under the dean 
of student work. Work is embedded in the philosophy of 
the college; all students are guaranteed a paid, on-campus 
work experience regardless of major or financial need. Ad- 
ditionally, the college lists 300 available positions, includ- 
ing off-campus jobs with schools, churches, and 
companies. A search of Berry’s own website reveals 105 
currently open positions with twenty-five employers, a 
third of which are located off campus. The intent is for stu- 
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dents to take up jobs with tasks broadly related to the field 
of study. Jobs differ in responsibilities and pay, and it is 
expected that students will progress to more demanding 
positions as they move through their studies. Jobs are su- 
pervised by faculty and staff, who provide informal and 
formal evaluation of performance following a well- 
articulated set of guidelines. Criteria refer to work-place 
competencies. Berry has carried out an evaluation of stu- 
dent learning that tracks conventional metrics of retention, 
graduation rates, and activities of students who leave prior 
to graduation or after graduation (via the National 
CLearninghouse). The evaluation also considers other evi- 
dence of learning and development, including through 
work experience, via both supervisor evaluations and a 
more focused examination of resumes and portfolios of a 
small random sample of students.”! 


@ Learning, Labor, Service at Berea College. A small private lib- 
eral arts college in Central Kentucky, Berea applies an ap- 
proach that obliges all students to work while pursuing 
studies. “Learning, labor, and service” in practice means 
that every student contributes to the college and the wider 
community through structured work. Work is part of the 
student experience, with expectations for performance and 
development that are evaluated for the record. Work may 
contribute to the operations of the campus, and can be un- 
derstood in context: Berea admits only students with dem- 
onstrated financial need, and students pay nothing 
out-of-pocket toward their studies and room and board. 
The combination of earnings from a substantial endow- 
ment, annual gifts, and financial aid from various sources 
— along with student-provided services — makes such an 
arrangement possible. The work assignments also include 
service learning, with agencies in the region. Students may 
take up work assignments within their program of study, 
or in related service opportunities (e.g., the Artifacts and 
Exhibit Studio at the Appalachian Center). The work expe- 
riences come under a designated program (headed by a 
dean), are supported by classroom instruction, and con- 
tribute to growing levels of competence and responsibility 
among student workers over the undergraduate years. 


@ Royal Melbourne Institute of Technology, Australia. From its 
origins, the Royal Melbourne Institute of Technology 
(RMIT) has been recognized for its emphasis on education 
for employment. In 2002, an estimated 7,000 RMIT stu- 
dents had work-based experiences in the course of their 
studies.” RMIT is unique in that it incorporates within one 
institution, technical/ vocational programs (coming under 
the Technical and Further Education (TAFE) division) and 
bachelor’s, master’s, and doctorates (under the higher edu- 
cation division). While some twenty-four schools (facul- 
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ties) come under these two divisions (twenty-one higher 
education; three TAFE), the schools are assigned to one of 
three main academic areas (organizationally called col- 
leges): Business; Design and Social Context; and Science, 
Engineering and Health. The opportunity for students to 
move from TAFE programs into higher education degree 
studies is promoted.” Also in Australia, a fully integrated 
“dual-degree” is available in community health offered 
jointly by the University of New England and TAFE New 
England. Students in the program participate in manda- 
tory work-based learning modules, and graduate with a 
vocational certificate, a vocational diploma, and a higher 
education degree.” 


4. Assessment 


Although assessments are not a tool to expand opportunities, 
they are a means to identify and expand good opportunities. Simply 
measuring work-based experience (as in Australia) brings greater 
attention to the issue in policy debates. Tennessee uses funding 
directed to campuses, offering grants to support academic pro- 
gram development that incorporate research-backed “high impact 
practices,” including applied learning.”> Because work-based 
learning across different campuses and initiatives has different 
goals, there is not necessarily a single standard for assessment. In 
fact, students and employers both may appreciate the value and 
use of alternative forms of assessment, which identify and refer to 
knowledge, skills, and attitudes not typically assessed in on- 
campus coursework. The Educational Testing Service is examin- 
ing noncognitive qualities to judge the “fit” between a student 
and a potential employment opportunity. The Council for Assess- 
ment of Experiential Learning offers assessments based not only 
on tests but also review of portfolios and other means. More as- 
sessments, and more sophisticated assessments, have the poten- 
tial to allow systems and campuses to provide better experiences 
for students and a better fit for employers. 


m “Fitness for work” assessment, Educational Testing Service. 
Within a TAACCCT grant supporting training for dis- 
placed workers at a consortium of twenty-three SUNY 
community colleges, ETS is rolling out an assessment of 
work readiness. The assessment concerns noncognitive 
dispositions that can be used to judge fitness for work, and 
more specifically how closely the learner aligns with the 
work-style of enterprises that might wish to make a hire. 
The assessment is more nuanced than conventional under- 
standings of generic behaviors (e.g., punctuality), and so 
offers a better picture of both what learners acquire inside 
and outside of the classroom, and also of the attributes em- 
ployers seek when filling positions. 


™ Council for the Assessment of Experiential Learning (CAEL). 
CAEL offers assessments of learning, wherever acquired. 
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Such assessments rely on tests, review of portfolios or 
other means, and in some instances are intended to enable 
judgments for recognition of experiential learning for aca- 
demic credit. Assessments of this type are not new, or un- 
usual. CAEL itself has been active since 1974. States have 
developed and implemented initiatives to bring about 
wider use of prior learning assessments, through man- 
dates, targeted funding, or articulation policies.”° What 
distinguishes CAEL from Advanced Placement, Col- 
lege-Level Evaluation Program (CLEP), and Excelsior Col- 
lege Examinations is the portfolio assessment. CAEL 
works with colleges and universities to assess prior learn- 
ing for academic credit, and also with employers to, in 
part, provide a profile of the knowledge, skills, and atti- 
tudes of employees.”” 


5. Points of Leverage at SUNY 


The examples of points of leverage just described suggest a 
broad range of strategies to extend and support work-based ap- 
plied learning experiences. Some of these strategies SUNY has al- 
ready adopted, others it has not. For starters, SUNY may be 
moving toward making applied learning a requirement for degree 
completion. New York Governor Andrew Cuomo wants to make 
experiential learning a requirement for SUNY students, a senti- 
ment echoed in Chancellor Zimpher’s 2015 State of the University 
address: “every SUNY degree will include an applied learning or 
internship opportunity as a prerequisite to graduation.” ”* The 
challenge, then, is to identify and provide the kinds of supports 
and leverage that might best encourage expansion of such 
experiences. 

In terms of incentives for students, the vast majority of place- 
ments throughout SUNY Works pilot campuses remain unpaid. 
Nonetheless, some schools have attempted to expand the total 
number of paid internships through their involvement with 
SUNY Works. Most notably, FIT used funds from Lumina Foun- 
dation to create nine $1,000 summer internships. These opportuni- 
ties were directed specifically at students unable to complete 
internships due to economic hardships. FIT is currently working 
with its development team to find grants to continue this program 
beyond the SUNY Works initiative.” 

With respect to employer buy-in, most Phase I and Phase II 
campuses are developing initiatives largely at the department 
level. At Brockport, Buffalo, Canton, Cortland, Erie, Niagara, and 
Stony Brook, for example, partners are identified by faculty, stu- 
dents, and career offices. Institutional outreach efforts such as ca- 
reer fairs (Albany) and alumni connections (Binghamton and 
Cornell) are also utilized. Contributing to these efforts, Chancellor 
Nancy Zimpher recently announced a plan to engage the CEOs of 
every Fortune 500 company with a presence in New York, as well 
as other large employers, to help promote the expansion of SUNY 
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Works.*° Consistent with Governor Cuomo’s START-UP NY eco- 
nomic development plan, the chancellor is also working to “estab- 
lish a Co-Op Task Force comprised of business leaders from 
across the state with a focus on regional Chambers of Commerce 
and companies coming to New York.”! 

Although, to the best of our knowledge, SUNY has not lever- 
aged work-study programs or state-funding, it has developed as- 
sessment tools. Building assessment into expansion, to see what 
works and therefore what ought to be furthered, is an important 
part of scaling up. A vast majority of Phase I and Phase II schools 
have developed a wide range of assessment tools to monitor stu- 
dent performance and learning outcomes. These typically include 
employer satisfaction surveys, reflective journal entries, and final 
papers. At Onondaga Community College, for example, employ- 
ers are asked to rate each student’s work-related skills (i.e., indus- 
try skills, organizational skills and work ethic) on a scale of one to 
five with one being the lowest performing indicator and five be- 
ing the highest.*? Similarly, Stony Brook asks employers to rate 
each student’s workplace etiquette, communication skills, and job 
performance on a scale of one to five with one being unacceptable 
and five being exceptional.*? Several schools have also developed 
assessments to monitor the overall quality of placements. At FIT, 
for example, students evaluate the worksite experience as well as 
the curriculum. With respect to the former, students are presented 
with questions like: 


m Was your orientation/ initial training adequate? 

m™ Did you have a clear understanding of what was expected 
of you? 

m™ Did you use your storehouse of knowledge and skills re- 
lated to your major? 


™ Did your sponsor organization hire you?™ 


With respect to the latter, students are asked whether class- 
room instruction helped them to assess their own personal skills, 
talents, values and interests, identify career possibilities, and write 
effective resumes.® Despite the prevalence of student and em- 
ployer satisfaction surveys and other forms of assessment, these 
practices vary from campus to campus. 

Although there are many points of leverage that can be en- 
gaged to scale initiatives up, as we have mentioned in this report, 
it is crucial to understand what initiatives are effective to make 
good decisions about what should be scaled up. There are many 
ways to define effective: increasing student satisfaction, better as- 
sessments of student qualification for jobs, or better academic out- 
comes, to name a few. In the following section we report on a 
pilot study undertaken to evaluate the use of administrative data 
to estimate the effects of work-based applied learning. In particu- 
lar, we look at whether work-based learning experience has an ef- 
fect on student retention, graduation, and employment at a single 
SUNY campus. 
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Rockefeller Institute 


Part Ill: Pilot Study 


Campuses participating in Phase I of Lumina-supported 
SUNY Works initiative had little or weak evidence of the effects of 
student participation in work-based applied learning on retention, 
graduation and eventually employment outcomes.*¢ The problem 
is a general one: campuses and their academic program units have 
only in recent years mounted more systematic and ongoing efforts 
to track student progress to graduation. Further, curriculum op- 
tions (such as applied learning) tend to be examined in one-off 
studies. Until fairly recently with the emergence of digital records 
and reporting and internet-based survey options, campuses and 
systems have not routinely and systematically tracked graduates 
into the postgraduate period. Under these circumstances, the use of 
extant administrative information to track students and graduates, with 
more finely grained analyses of differences in progress, completion, and 
employment according to curriculum options such as work-based applied 
learning, opens up potentially useful information that can be obtained 
easily, quickly, and routinely. Recognizing both the lack of informa- 
tion and the potential, the Baseline Report included a recommen- 
dation to undertake an exploratory study to test the feasibility of 
using extant academic unit records, also linked to NYS Depart- 
ment of Labor UI wage records, to estimate such effects. Under 
the present project, we have carried forward this 
recommendation. 

There are several advantages of an exploratory study. First, 
the design of the study obliges attention to methods and mea- 
sures, and so can surface needed details on applied learning expe- 
riences that are not presently collected. Second, the design 
developed for the pilot serves as a method that might be adopted 
for use at campus and system level. Third, the pilot provides a 
first test of the use of information on graduates employed in New 
York State. Although not new in a number of states, the possibili- 
ties for linking SUNY academic unit records with NYS Depart- 
ment of Labor UI wage records opened up only this year. SUNY 
has undertaken a few studies using linked UI wage record infor- 
mation, and this pilot study has relied on SUNY’s vice president 
for institutional research to pursue the linked data. 

The pilot study aims to estimate the effects of participation in 
work-based applied learning at one SUNY campus, which was se- 
lected because of existing data resources. The Rockefeller Institute 
project team worked closely with two campus officials with very 
close knowledge of the types of applied learning experiences of- 
fered in particular degree programs and also with expertise and 
familiarity working with academic unit records and linked UI 
wage records. The project also secured advice from an expert with 
experience working with linked academic unit record-wage re- 
cord data in other states as well as experts in the design of assess- 
ment of learning studies.*” 
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Researchers have used academic unit records to understand 
the effects of work-based experience on academic results, such as 
grades, credits earned, and retention to the second year.*§ While 
research using linked academic — wage record data bases has 
grown over recent years, these studies are more particular, exam- 
ining differences by campus for degrees in a certain field and a 
particular levels. This pilot study, by contrast, is distinctive in its aim 
to apply linked unit record data to estimate the more finely- grained aca- 
demic and employment effects of work-based applied learning within and 
across degree programs. For this reason, the considerations in the 
design, metrics, and implementation are presented in some detail 
along with the difficulties arising. The difficulties stand as impor- 
tant limitations. Given that academic unit records typically lack 
details on curriculum options such as applied learning experi- 
ences, substantive involvement of knowledgeable campus-level 
staff will be required to carry out such analyses in the near term. 


A. Design, Metrics, and Implementation 

The pilot sought to explore how far — and how — student 
unit records can be used to both track academic results and em- 
ployment outcomes (the latter through matches with the 
NYSDOL UI wage records). The pilot had as its purpose to ex- 
plore the feasibility of such an approach, to lay out the kinds of 
metrics that could be used, and to identify limitations and gaps. 
What is learned from a pilot advanced at campus level should re- 
veal the potential — and limitations — of the approach for other 
campuses and SUNY Works. 


1. Design 


Isolating the impact of work-oriented learning experiences: The pi- 
lot study was conceived as a method to gauge the effects of 
work-oriented applied learning experiences on retention, gradua- 
tion, and employment outcomes. To estimate effects, the academic 
and employment outcomes of participants in such experiences 
need to be compared to the outcomes of students who did not 
participate. Since the students in these two groups may differ in 
ways other than participation in work-based applied learning ex- 
periences, the method used ideally should take into account the 
extent to which other attributes are associated with retention, 
graduation, and employment outcomes. 

With expert advice, we considered different options. The most 
appropriate for the pilot study is a “matched” students strategy. 
Under this approach, academic and socioeconomic information is 
used to “match” students, i.e., to identify students who we judge 
to have the same propensity to participate in a work-based ap- 
plied learning experience. The academic and employment out- 
comes for those who participated in the applied learning 
experience are compared to those in the “matched” group of stu- 
dents to come up with an estimate of the effects. Several relevant 
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studies have used such a method to come up with estimates of the 
effects of applied learning.®? 

Working with the participating SUNY campus, we explored 
the information available on the academic unit records and the UI 
wage records. For the analysis (and, subsequently, interpretation 
of results), several considerations and limitations were identified. 

Work-based learning opportunities differ in structure and re- 
quirement, which presents challenges for overall assessment of 
impact.” 

@ Student records do not distinguish between different types of 
work-based learning experiences. SUNY Works distinguishes 
among eight categories of work-based applied learning. 
For the SUNY campus participating in the pilot study, stu- 
dent academic records do not contain sufficient informa- 
tion to distinguish among these categories or, for the 
purposes of the pilot, on characteristics of the applied 
learning experiences that might be taken into account in 
assessing effects. That is, differences in the effects of 
work-based applied learning experiences may vary owing 
to characteristics of the particular experience. Clinical in- 
ternships, for example, tend to be highly structured, 
closely monitored, and required in fields that lead to li- 
censing. Those characteristics do not apply for all intern- 
ships. In view of the absence of such information in 
student records, the Rockefeller Institute project team 
worked with colleagues at the participating SUNY campus 
to identify a set of fields of study for which opportunities 
for work-based applied learning experiences are offered as 
an option (not a requirement) and are likely to be more 
similar with respect to academic expectations and student 
interests. We introduce more detailed comparisons of ef- 
fects by field in the analyses of employment outcomes. For 
the purposes of the pilot, the applied learning experiences 
are internships. 


m Many student experiences are not recorded. An important lim- 
itation in the available data is uneven coverage, such that a 
good share of students undertake work-based learning ex- 
periences (e.g., internships) outside of those recognized 
through academic programs. The extent of such participa- 
tion in such experiences at the pilot campus is not known. 
A recent survey of graduates suggest that the proportion 
with some work-based learning experience while in col- 
lege is around 35 percent.®! One implication is that the use 
of available information in student academic records may 
lead to an underestimate of the effects of participation in 
work-based applied learning experiences, as some stu- 
dents in the “match” group may have such experiences 
and so manifest retention, graduation, and employment 
outcomes more similar to students with recorded 
work-based applied learning.” 
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2. Metrics 


The intended outcomes of participation in SUNY Works is in- 
creased retention, graduation, and employment. The academic 
unit records alone and linked to UI wage records are well-suited 
to examine these outcomes, but they are not perfect. The 
Rockefeller Institute project team consulted with experts and re- 
viewed extant studies on the types and range of measures to be 
constructed. 

Although we employed academic unit records from a single 
campus to generate estimates of the effects of work-based applied 
learning experiences on retention, graduation and employment 
outcomes, additional measures may better capture the complexity 
of academic pathways, student experiences, and wide range of 
potential effects. 


@ Academic unit records at a single campus do not capture stu- 
dents who complete their studies at another institution, which 
may overstate the effects of work-based learning. For the pilot 
study, information on retention through to graduation is 
limited to student registrations at the participating SUNY 
campus. Students who do not appear as registered in suc- 
cessive years (or appear as a graduate) may continue and 
complete their studies at another college or university. 
Prior research suggests that the frequency of such path- 
ways is substantial. Although beyond the scope of the pi- 
lot study, students who transfer out can be tracked 
through other data sources (SUNY, National 
CLearninghouse, and through multistate data sharing, 

e.g., the Western Interstate Commission for Higher Educa- 
tion, WICHE).*4 From the perspective of the system, the es- 
timates produced in the pilot study may overstate the 
effects of work-based applied learning on retention, gradu- 
ation, and employment outcomes (the latter, with the pres- 
ent design). Preliminary descriptive analyses of 
year-to-year retention reveal high and similar persistence 
before, during, and after any (or no) work-based applied 
learning experiences. A somewhat larger difference be- 
tween those with an internship at any time and those with- 
out internships (simple descriptive statistics) is apparent in 
graduation rates. 


™ Nonacademic effects, such as student engagement and student 
satisfaction, may be a significant outcome of work-based learning 
experiences. Researchers consider measures of effects on 
students other than retention or graduation. A strong line 
of inquiry focuses on student engagement, and the extent 
to which differences in engagement are associated with 
both practices at colleges and academic results. Two ex- 
amples have been mentioned above: the Australasian Sur- 
vey of Student Engagement (modeled on NSSE in the US) 
focuses specifically on the student experience in work-inte- 
grated learning; SUNY’s TAACCT grant supporting stu- 
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dents and programming at twenty-three community 
colleges has incorporated an assessment of “job readiness” 
to identify noncognitive capacities that complement the 
content knowledge to be acquired through coursework. 
Such capacities may be developed also through 
work-based applied learning experiences, although evi- 
dence on the ways in which these and other noncognitive 
attributes figure into retention and graduation is limited. 


While the UI wage records linked to academic unit records are 
an important way to measure employment (through wages) in 
New York State, here, too, there are considerations and 
limitations. 


m The wage records provide information on formal employment 
within New York State, and do not show self-employed, em- 
ployed persons outside New York, or students who continue in 
graduate programs. Employers are obliged to file, each quar- 
ter, a wage record for every employee. According to the 
NYS Department of Labor, more than 97 percent of those 
working in the state appear in these records; but self-em- 
ployed individuals are excluded. In comparison, for 
CUNY, more than 80 percent of associate and bachelor’s 
graduates were employed (and showed up on wage re- 
cords) during the period 2003 to 2010.” The proportions in 
this pilot study may be lower and vary substantially, de- 
pending on the period covered, field of study, and region. 
Information from other states as well as for SUNY, suggest 
that as many as half of graduates may not appear on 
NYSDOL wage records within the first four to six quarters 
following graduation. The proportions coming up on the 
records are higher for community college graduates and 
lower for university graduates.°* Some may enter graduate 
school, others may move out of state for employment. The 
destinations for those NOT appearing in wage records is 
not known (but see WICHE for a multistate tracking of ac- 
ademic and employment destinations). For the purposes of 
the pilot study, a measure of the proportion of graduates 
employed in New York State is relevant and appropriate 
for estimates of the effects of work-based applied learning 
experiences. Nonetheless, the extent to which such experi- 
ences make it more likely that a graduate will work as op- 
posed to continue into graduate education is not known. 
The finding that students participating in applied research 
experiences as undergraduates appear more likely to pur- 
sue graduate study may suggest that participation in 
work-based experiences could make it more likely that 
graduates enter employment.” 


@ More reliable measures of employment occur within four to six 
quarters after graduation. Quarterly wage records may pick 
up transitions into positions, especially in the early 
postgraduation period. For this reason, the measures of 
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employment outcomes refer to each matched graduate’s 
wage records within six quarters following graduation. 
The three proposed measures of employment outcomes, 
specifically, are: employed within New York State in any of the 
first six quarters following graduation; the highest wage re- 
corded in any of the first six quarters following graduation; and 
the number of quarters until the first match (wage record) up to 
the first six quarters following graduation. 


@ Some researchers have used alternative measures of employment 
outcomes. Purdue University’s collaboration with Gallup to 
survey graduates has explored student assessments of the 
relevance of their studies for the jobs they hold, job satis- 
faction, and the extent to which internships and coopera- 
tive learning allowed them to apply what was learned in 
the classroom.! The previously mentioned assessments 
developed under the TAACCCT grant assesses “work 
readiness” on dimensions that align with what supervisors 
describe as ideal attributes in a hire. 


3. Implementation 

Work with academic unit records, and in particular linking 
these records to NYSDOL wage records, required a sequence of 
steps. The main considerations, as well as a key limitation, are 
identified here. 


® Incollaboration with experienced and knowledgeable staff 
at the participating SUNY campus, Rockefeller Institute 
project staff developed the field of inquiry to be entering 
cohorts for 2006, 2007, and 2008. The three cohorts, when 
first enrolled, had 2,708, 2,760, and 2,880 students, respec- 
tively, a total of 8,348 students, which yielded a pool of 
students of a size sufficient to yield the sample for the 
analysis. The enrollment and internship information for 
these students were provided up until the Fall term of 
their seventh academic year (2012, 2013, and 2014 for en- 
tering cohort 2006, 2007, and 2008 respectively) enrolled at 
the campus. Students may have graduated, transferred 
out, or dropped out before this last term. The demographic 
and socioeconomic characteristics of the sample are sum- 
marized in Appendix C, Table 3.1. Descriptive statistics on 
college preparedness indicators, including Student Apti- 
tude Test (SAT) math score, SAT critical reading score, and 
high school grade point average (GPA) are shown in Ap- 
pendix C, Table 3.2. Within the time available, the ap- 
proach adopted was to work with these cohorts to 
examine retention, graduation, and employment out- 
comes. Estimates of effects on retention and graduation 
could have been made more current through use of more 
recent cohorts (entering, say, in 2010). Specific analyses of 
effects on employment outcomes required earlier cohorts. 
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™ Given the limited information on work-based applied 
learning experience in the academic unit record, we relied 
on advice from colleagues at the participating SUNY cam- 
pus concerning the types of internships and co-op learning 
(most particularly, requirements if any) across fields of 
study. Those exchanges led to decisions on the sub-set of 
students to include in the analysis. 


m Analyses of retention and graduation was carried out by 
Rockefeller Institute staff. 


m@ The transfer of campus-level records had two steps. In the 
first step, a file with student records identified with a ran- 
dom ID was provided to Rockefeller Institute staff. The 
random IDs for the subsample of graduates selected 
through analysis at RIG, with codes appended to desig- 
nate groups for the comparisons — “interns” and 
“noninterns” for each of a set of combinations of back- 
ground characteristics and fields of study — are sent back 
to the campus. In the second step, the campus attaches the 
student’s Social Security Number (SSN) to enable the link 
to NYSDOL wage records, and sends these records to 
SUNY System which has established the formal request for 
the transfer of the file to NYSDOL. 


™ Under implementation of the present Memorandum, 
NYSDOL does not provide linked individual wage re- 
cords. Further, NYSDOL does not provide information 
(cell counts or means) for any cell with fewer than ten 
cases. With these considerations in mind and also adopt- 
ing a conservative estimate that at least half of the SSN’s 
sent to NYSDOL will not be matched (graduate in gradu- 
ate school, moved out of state, not employed in a covered 
position), we established groups, identified by codes on 
the file to be submitted to NYSDOL, of sufficient size to 
clear the cell count threshold minimum. 


® NYSDOL will return tabled data to SUNY, subsequently 
conveyed to Rockefeller Institute for submission to SUNY 
Works. [Step not yet completed. ] 


B. Analysis and Findings 


We use descriptive data, bivariate analyses, and multivariate 
regressions (more specifically, a series of logistic regressions) to 
reveal the relationships between participation in an internship 
and the demographic, socioeconomic, and academic backgrounds 
of students, and learning outcomes, including retention, gradua- 
tion, and labor market outcomes. 

As mentioned above, associations do not always reveal causal 
relationships: Even if participation in an internship is associated 
with a higher probability of retention, graduation, employment, 
and earnings, the associations may mask other causes for favor- 
able outcomes. Among statistical approaches used to capture the 
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effects of participation in internships, propensity score matching 
(PSM) is a method that has been applied with respect to 
work-based or discovery experiences.!°! The PSM approach takes 
two steps to identify the causal effects of interest. First, available 
observable attributes are used to calculate the propensity to par- 
ticipate in a treatment, which, in our analyses, is participation in 
the internship. An individual who took an internship is matched 
to another who did not take the internship (in the control group) 
but had the same “propensity” to take the internship based on 
other attributes. This step is an approximation to a randomized 
controlled trial (RCT). It addresses the problem of extrapolation 
that often exists in ordinary least square (OLS) regressions! and 
guarantees that apples are only compared to apples. Second, the 
difference in outcomes between the treatments and controls — in- 
ternship/no internship for each matched pair (or matched group) 
is calculated and then averaged among the matched pairs 
(matched groups) to get an estimate of the difference in the learn- 
ing or labor market outcome of interest. However, the causal esti- 
mates using PSM are only valid to the extent that we capture the 
most important observable attributes of students. Available infor- 
mation on relevant attributes, such as parents’ education, family 
income, and so forth, is limited. Moreover, we do not have 
measures of important, but unobservable attributes, such as 
motivations. 

These limitations raise a second area where further informa- 
tion will be needed to make full and appropriate use of such 
methods in the future (the first being program details on intern- 
ships). Even with additional information, attention will need to be 
directed to group sizes sufficient to permit finely grained analyses 
under PSM. Put simply, interest in generating estimates of the ef- 
fects of internships by feature and field for use at campus level di- 
rects attention to smaller groups while appropriately 
implemented PSM techniques require larger numbers. Strategies, 
such as those adopted for the project to draw from several years’ 
cohorts, will need to be identified and evaluated. 

A further consideration is identification of the appropriate 
comparison groups. For estimation of the effects of internship on 
retention, full analyses are limited to the sub-sample of students 
retained to the prior year. For estimation of the effects of intern- 
ship on graduation, full analyses are limited to the sub-sample of 
students retained to Spring semester of year four (for year four 
graduation).! For estimation of the effects of internships on em- 
ployment outcomes, analyses are limited to the sub-population of 
graduates. 


1. The Scale of Internship Participation 


Academic records do not capture all internship experiences. Just 
over 17 percent (or 1,435 out of the 8,348) of entering students 
took at least one internship by the beginning of the Fall term of 
their seventh academic year, or the Fall term of the seventh year, 
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at the participating campus. As mentioned, this number is 
roughly half of the 35 percent of college graduates that Gallup re- 
ports.1°* However, Gallup counts all self-reported internships, 
while the academic records that we are working with capture only 
those internships that are credited within study programs. 
Students who took advantage of internships most often completed 
them late in their academic career. The most common time to take an 
internship was the Spring and Fall terms of the third and fourth 
academic years (see Appendix C, Table 3.3 and Figure 3.1). 


2. Participation in Internships — Background Characteristics 
and Participation 


Females, African Americans, those whose tuition residency in first 
term was not foreign, and those with relatively lower SAT scores (both 
math and critical reading) were more likely to participate in at least one 
internship during their studies at the participating SUNY campus. 
These associations are statistically significant at .01 or .10 levels. 
However, although the differences are statistically significant, the 
magnitude of the differences in participation in internships 
among the groups is modest. Further, rough measures of re- 
sources (whether students were offered the Pell Grants) and abil- 
ity (high school GPA) are not associated with internships taken 
(Appendix C, Tables 3.4 and 3.5). Two attributes show differences 
of some magnitude: gender and ethnicity. The proportion of in- 
ternship taking for female students is almost 10 percentage points 
higher than that for male students. The proportions for Asian, Af- 
rican American, and Hispanic students are 4.7, 9.5, and 5.4 per- 
centage points higher, respectively, than that for white students, 
while American Indian/ Alaskan Native students almost 4 per- 
centage points lower than white students. 

We used logistic regression to further estimate how the proba- 
bility of taking at least one internship recorded in the student’s 
unit records is associated with background characteristics. The re- 
sults reveal similar patterns of internship participation by gender 
and ethnicity, but show somewhat different patterns of internship 
participation by Pell and academic performance. Our baseline 
group consists of female white students who entered college in 
2006, weren't offered Pell Grants, had average academic perfor- 
mance, and whose tuition residency in the first term was New 
York State. According to our estimation, 16.5 percent (the proba- 
bility shown in the intercept row of Appendix C, Table 3.6) stu- 
dents of the baseline group would take at least one internship in 
college. Compared to the baseline group and when all other char- 
acteristics are unchanged, the estimated probability for male stu- 
dents to take at least one internship in college is 9.7 percent, or 6.8 
percentage points less. The difference is statistically significant at 
the .01 level. The attributes that contribute to a difference in the 
likelihood of participation in at least one internship (all attributes 
taken into account) are racial/ethnic background (higher proba- 
bility for Asian, Black, or Hispanic) and SAT scores (lower 
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probability for higher SAT scores). With respect to the latter, for 
example, students whose SAT Math score is one standard devia- 
tion above the mean score are about 2 percentage points less likely 
to participate in at least one internship significant at the .01 level). 

Further, the logistic regression reveals relationships that are 
not apparent in the bivariate associations. Students with Pell 
grants were a little less than 2 percentage points (1.7%) less likely 
to take at least one internship, and students whose high school 
GPA is one standard deviation higher than the mean are about 1 
percentage point less likely to participate in internships. The size 
of these differences are not substantial, suggesting at least on 
those attributes that we can observe, the profiles of students tak- 
ing internships are not much different than those who do not. 

We note, from descriptive statistics, that participation in in- 
ternships varies substantially by field of study. This study pro- 
gram characteristic is taken into account in analyses of 
employment outcomes (below). 


3. Participation in Internship and Retention 

Internships are associated with greater retention. Those who took 
at least one internship were more likely to be retained than those 
who never took an internship (see Appendix C, Table 3.7). 

It is difficult, however, to disentangle the effect of the intern- 
ship on retention from other factors. Using logistic regression 
models, which provide estimates of the odds of being retained to 
the end of the third and fourth academic years, participation in at 
least one internship by the end of the third academic year was 
positively associated with retention at the end of the year even af- 
ter controlling for gender, ethnicity, and academic performance 
before and after enrollment in college. 

Students who took at least one internship by the end of the third year 
were 3 percentage points more likely to be retained at the beginning of 
the fourth academic year than those who never took an internship (statis- 
tically significant at the .01 level). What should be noted is that we 
only included in the analysis those students who were retained 
and not graduated at the beginning the third year to calculate the 
retention rate at the end of the third year. Thus, the retention rate 
for our baseline group is already as high as 96.8 percent. With this 
baseline in view, the 3 percentage point difference is quite impres- 
sive. Here the baseline group consists of those who didn’t take 
any internship during their third year and who were female white 
students, entered college in 2006, weren’t offered Pell Grants, had 
average academic performance, and whose tuition residency in 
the first term was New York State. However, the results are not as 
impressive for retention in the fourth year. Students who took at 
least one internship by the end of the fourth year were only 0.3 
percentage points more likely to be retained at the end of the 
fourth academic year than those who never took internship had 
(not statistically significant. Please see Appendix C, Tables 3.8 and 
3.9 for more details). 
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Using the method of propensity score matching (PSM) to esti- 
mate the effects of internship on retention to the end of year three, 
our results show that students who took at least one internship by the 
end of the third year were 2.8 percentage points more likely to be retained 
at the end of the third academic year than similar students who never took 
an internship (statistically significant at the .01 level). Students who took 
at least one internship by the end of the fourth year were only 0.4 percent- 
age points more likely to be retained at the end of the fourth year than simi- 
lar students who never took an internship (statistically insignificant at 
the .10 level). The results are shown in Table 3.11. The procedures 
and more detailed results of the statistical tests carried out are pro- 
vided in the Appendix. These results are quite similar to those ob- 
tained in the logistic regressions already discussed. 


4. Participation in Internships and Graduation 


Students who take at least one internship have better graduation 
rates than those who do not take internships. To isolate the effect of in- 
ternship on graduation only (i.e., removing the effects of intern- 
ship on being retained to the year at which graduation is 
evaluated), we limited the analysis to students retained to the sec- 
ond half of the fourth year. Of note, about 60 percent of graduates 
from the three cohorts graduated in their fourth year (with most 
of the remainder graduating later). 

For those students and fitting a logistic regression model to 
also control for other student characteristics that may be associ- 
ated with differences in graduation rates, we find that students 
with internships were 13.6 percentage points more likely to grad- 
uate than those in the baseline group (results significant at .01 
level; see Appendix C, Table 3.10). Again, the baseline groups con- 
sist of those who did not take any internship until the end of their 
fourth year and who were female white students, entered college 
in 2006, were not offered Pell Grants, had average academic per- 
formance, and whose tuition residency in the first term was New 
York State. 

Applying the method of propensity score matching, we find 
that students who took at least one internship by the end of the Spring 
semester of the fourth year were 9.5 percentage points more likely to 
gradate than similar students who never took any internship but were 
still enrolled in that semester (statistically significant at .01 level). 
The results are shown in Appendix C, Table 3.12. The procedures 
and more detailed statistical results are provided in the Appendix. 
These results are somewhat lower than those obtained in the lo- 
gistic regressions already discussed. 

Student program characteristics, such as field of study, are not 
included in the analyses leading to the estimates. Owing to differ- 
ences in both requirements for and/or expectations within aca- 
demic programs, the internship taking — and the effects of 
internship taking on graduation in a given year — may well differ 
by field. We take into account field of study in the detailed analy- 
ses of employment outcomes (below). 
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5. Participation in Internships and 
Labor Market Outcomes (to be Completed) 

Because the New York State Department of Labor will only 
provide “de-identified” data that strip records of identifiers (name 
or social security number) and further restricts minimum cell size 
to 10, we continue to work through arrangements to ensure suffi- 
cient cell sizes and meaningful results. Once those arrangements 
are fully implemented and the requested analyses are completed 
by NYSDOL, the results will be submitted to SUNY. 
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Part IV: Conclusion 


Applied learning is a strategy now being more widely used to 
improve student learning, retention, graduation, and employment 
outcomes. The relevant experiences identified and described in 
this report demonstrate recent efforts at system and campus levels 
to extend opportunities. They generally show strong performance 
with learning results and employment outcomes in systems or on 
campuses with considerable experience. The initiatives we identi- 
fied all aim to engage more — if not all — students in applied 
learning experiences, including work-based opportunities similar 
to those in SUNY Works. If nothing else, this much is clear: Al- 
though SUNY Works (like SUNY) may be unique with respect to its 
reach across types of fields and institutions, it is not alone in seeking to 
provide applied learning opportunities, work-based learning, in particu- 
lar, to all students. 

Yet, the very breadth of study programs and student learners 
in SUNY is also the main challenge in identifying and expanding 
work-based applied learning experiences. Still, the campus- or 
system-level initiatives that we reviewed have counterparts 
within SUNY. Efforts and success in providing applied learning 
opportunities for all students can be found elsewhere for each de- 
gree program, level of institution, and type of student within 
SUNY. The experience elsewhere is relevant, even if SUNY’s aim 
is to do more by spanning all study programs, sectors, and 
students. 

Our review and discussions with knowledgeable individuals 
at system and campus levels lead us to draw attention to an equal 
challenge. Implicit in the expansion of opportunities for 
work-based applied learning opportunities for all students is the 
requirement that the applied learning experience is worthwhile. 
Both long-standing system initiatives, such as the German dual 
system or the Swiss Universities of Applied Sciences, and campus 
initiatives, as those found at Northeastern, Berea, Berry, Cal Poly 
and South Dakota School of Mines, reflect common features that 
support learning through a student’s work experiences and, in 
this way, allow such experiences to make their greatest contribu- 
tions to academic results and employment outcomes. 

We suggest a few key features are likely to be particularly im- 
portant: structured learning integrated in the curriculum; work- 
shops or seminars that complement and support the work-based 
learning experiences; sustained engagement in such experiences 
over the study program; relevant and useful assessments; and ac- 
ademic credit. These features do not come from a synthesis of the 
research literature on student learning, but rather from a much 
more selective consultation of that literature, the initiatives that 
we identified and reviewed, and the discussions with those 
knowledgeable at system and/or campus levels in this field. We 
offer them as suggestive of the kinds of features that will need to 
be incorporated and reflected in all work-based applied learning 
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opportunities as SUNY Works seeks to extend such opportunities 
to all students. 

Worthwhile work-based applied learning extended within 
and across campuses requires specific attention in mission and 
changes in ways of working. Changes such as these are difficult to 
make and sustain. They take time, but over time they can pick up 
steam. Allocated resources and conditions on the ground can sup- 
port such changes. These considerations, drawn from our review 
of experiences elsewhere and conversations with those knowl- 
edgeable about the changes of the type envisaged across SUNY, 
provide a basis for targets for action in SUNY Works. 

To bring about expansion of worthwhile work-based learning 
opportunties, SUNY Works might focus efforts in three areas, 
where experience elsewhere provides examples of possible points 
of leverage. 


= Promote ways to better integrate applied work-based learning 
into study programs. As just suggested, such integration 
might include early and sustained engagement with 
work-based applications, workshops that support students 
as they move between academics and work-based experi- 
ences, useful assessments, and/or credit for work-based 
experiences. Faculty engagement should be encouraged, 
through infrastructure (as in Michigan’s Urop, which takes 
on administrative tasks that otherwise would fall on fac- 
ulty), incentives (such as Tennessee’s grants to support 
curriculum incorporating “high impact strategies,” among 
which structured work-based learning), and support for 
and recognition of faculty (as advanced through the Office 
of Learning and Teaching in Australia’s Department for 
Education and Training). While relevant experiences with 
mandates have been identified, these emphasize outcomes 
(Massachusetts’ Civic Learning policy), provide flexibility 
(University System of Maryland’s E&E policy), and are 
most specific when targeted on particular fields (Ger- 
many’s dual system, Switzerland’s UAS). An expansion of 
work-based applied learning can serve other SUNY-wide 
priorities put to campuses, whether access, retention, or 
graduation of students from under-represented groups 
(where Michigan’s Urop and proposals for state and fed- 
eral work-study support serve as relevant applied learning 
examples that would support such a priority) or 
“systemness” (where the Trade Adjustment Assistance 
Community College and Career Training — TAACCCT — 
is an an example of a work-based learning effort that al- 
ready engages a number of SUNY community colleges). 


mB Support campuses, individually and within regions, to develop 
relationships and engagements with employers. The initiatives 
identified reveal a wide range of means to do this, from 
building on relationships already established for economic 
development and workforce development (Photovoltaic 
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Manufacturing Consortium and TAACCCT, are examples) 
to broader cost sharing arrangements (as in purposeful 
state work-study or possible tax concessions). These means 
may oblige involvement of agencies outside SUNY and 
also open up new ways of working in partnership — span- 
ning campuses, as in the chancellor’s appeal to Fortune 
500 companies, but also to other companies or agencies 
with a presence throughout the state. 


= Help campuses make work-based applied learning attractive to 
students. Worthwhile student work experiences support 
student learning: providing a work experience is not the 
same thing as providing a work-based learning experi- 
ence. The long-standing initiatives we examined incorpo- 
rate workshops, seminars, and teaching to support and 
complement work experiences and generally integrate 
work experiences within study programs. To strengthen 
work experiences falling outside of study programs, SUNY 
might help to develop a means for assessment of that ex- 
perience, for campus credit or for more general value to 
the student to identify strengths and weaknesses. Such as- 
sessments (and/or recognition) might apply to work expe- 
riences for students working on campus or in part-time 
jobs locally that are not, at present, drawn into study pro- 
grams (the approaches at Berea and Berry may be rele- 
vant). Further, student participation can be incentivized 
with stipends, most directly through a more purposeful 
use of work-study (as advanced in discussions in Wash- 
ington state or proposal for Federal Work-Study coming 
forward for the reauthorization of the Higher Education 
Act). 


Taking initiatives to scale requires information on what works. 
The needed information base extends beyond research studies in 
the field, which may provide powerful evidence but are less use- 
ful in evaluation at campus or system levels or in informing stu- 
dents and employers. The pilot study carried out for this project 
shows both the potential for, and limitations of, using administra- 
tive records also linked to UI wage records for these purposes. 
The gaps to be filled are several: bring work-based experiences 
presently not recorded onto student administrative records; add 
details that distinguish among work-based experiences (likely go- 
ing beyond SUNY Works’ four categories of co-op, internship, 
work-study, and clinical); and develop means to assemble infor- 
mation on sufficient numbers of students by field and back- 
ground. These gaps, and other limitations, can be addressed with 
the help of campus staff and faculty, and in so doing, help to 
make data generated from such analyses even more useful. Even 
so, information for use at the academic program and campus level 
may need to be generated in other ways (e.g., supported experi- 
mentation). Further, the effects of work-based applied learning 
reach into areas of skills and attitudes that are less commonly 


Rockefeller Institute Page 54 www.rockinst.org 


Higher Education Applied Work-Based Learning at the State University of New York 


assessed in study programs. These effects are potential value to 
students and employers. New assessment might be developed 
and made widely available, for which the “Fitness for Work” as- 
sessment ETS has developed under the TAACCCT Grant in 
NYES, the experience at Berry College, and the use in Australia of 
a survey of student engagement that includes “work integrated 
learning” as a specific engagement scale represent examples. 

For SUNY Works, there may be value in encouraging the cre- 
ation of a venue through which interested and involved faculty, 
cooperating employers, and interested third-parties (among 
which, state agencies, sponsors, and SUNY staff) can identify 
questions and issues, present the results of careful studies for 
wider discussion and review, and otherwise participate in the 
evolution of work-based applied learning across SUNY. A model 
here is the Australian Collaborative Education Network. 

Taken together, the features and points of leverage that we 
identify and illustrate in this report provide a range of options for 
consideration, with adaptation, by SUNY. Existing work-based 
applied learning initiatives found on campuses throughout SUNY 
are often well-structured and include organized workshops, but 
they do not as often include work-based experiences that early in 
the student’s academic career, integrate those experiences into the 
core curriculum, and offer alternative assessments. Of course, par- 
ticular features in a given work-based learning opportunity will 
depend on the campus setting and field of study, and particular 
means to encourage and support such learning opportunities for 
all students may not be appropriate — or effective — in all in- 
stances. Our review suggests, however, that building opportuni- 
ties into the curriculum, making it an important part of what 
students at SUNY do, and working closely with stakeholders to 
make sure educational and other needs are met are keys to suc- 
cessful initiatives. 

There is much to build on: many students throughout SUNY 
presently participate in applied learning experiences, and SUNY 
already incorporates design features and points of leverage found 
in initiatives elsewhere. SUNY Works’ continuing efforts, we sug- 
gest, will benefit from support for the development of information 
on the effects of applied learning experiences, at campus and 
SUNY system level and relevant for use by students, employers, 
and faculty as well as officials and the wider public. Beyond this, 
there may be more to learn from the very wide range of experi- 
ences on campuses and within systems that have extended — or 
aim to extend — applied learning broadly. In these ways, SUNY 
Works can identify, adapt, evaluate, and refine means to advance 
the ambitious goal set by Governor Cuomo and Chancellor 
Zimpher: an opportunity for every SUNY student to have an ap- 
plied learning experience. 
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Appendix A: Survey 


Thank you for participating in our survey. Your feedback is important. The information that you and 
colleagues from around SUNY provide will be used to help SUNY system administration better sup- 
port the development of applied learning experiences for students across SUNY. 

PART A: Respondent 


Please answer the following questions regarding your institutional affiliation and position. 


1, What is the name of your institution? 
2 What is your current position? 
a How long have you held this position? 


o Two years or less 

o Three to five years 
o Six to ten years 

o More than ten years 


4. How long have you been on this campus? 
o Two years or less 
o Three to five years 
o Six to ten years 
o More than ten years 


D, What other administrative or academic positions have you held? 

No other positions 

A position similar to this one on this campus (e.g. different college) 

A position similar to this one at another campus 

A position not similar to this one, with any institution, organization, or employer 


° 


000 


6. In which offices or areas have you worked in your career to date. Check all that apply. 
Student Affairs 

Career Office 

Student Employment Office 

Financial aid office 

President or Provost Office 

Office of the dean of college or division 

Business and finance office 

Counsel 

Research office 

Development office 

Human Resources office 

Institutional Research, including assessment and accreditation 
Other position in institutional administration 

An academic position (faculty, teaching) 

Other position in higher education 

Other position outside higher education (please specify) 


o0o0o0oooo0o0oooaooocogdo 80 
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7 In which of the following applied learning programs do you have some involvement? Check all 
that apply. 

Internships 

Co-operative learning 

Service learning 

Community Service 

Clinical-teacher preparation 

Clinical placements — healthcare 

Research/entrepreneurship/field study 

Work study 

Other (please specify) 


000000000 


PART B: Applied Learning Programs on Campus 


8. For the program you know the most about, what type of program is this? 
Internships 

Co-operative Learning 

Service Learning 

Community Service 

Clinical-teacher preparation 

Clinical placements — healthcare 

Research/entrepreneurship/field study 

Work study 

Other (please specify) 


000000000 


9. How many years has this program been available to students? 
Not yet available 

Two years of less 

Three to five years 

Six to ten years 

More than ten years 

Don’t know/ Unsure 


00000 0 


10. Is this program among the oldest or among the newest applied learning programs on your 
campus? 

Not applicable (i.e. program is not yet available) 

Among oldest 

Among newest 

Don’t know/ Unsure 


° 


oo0°0 


dt. Where did this program get its start? 
o Within an academic or professional department 
Within a college or division on this campus 
At campus level 
Don’t know/ Unsure 
Other (please specify) 


0000 


12, Where is this program now located? 
o Within an academic or professional department 
o Within a college or division on this campus 
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o Atcampus level 
o Don’t know/ Unsure 
o Other (please specify) 


13. What were the primary reasons for the launch of this program? Check all that apply. 
Faculty interest 

Student demand 

Employer demand 

President or Provost interest 

External funding 

Additional staffing provided by the campus to establish such a program 

Don’t know/ Unsure 

Other (please specify) 


ooo0o0o00g00c 0 


14. Now, looking again at the primary reasons for the launch of this program, which ONE would 
you consider to be the MOST important reason? Check one. 
o Faculty interest 

Student demand 

Employer demand 

President or Provost interest 

External funding 

Additional staffing provided by the campus to establish such a program 

Don’t know/ Unsure 

Other (please specify) 


oo0o0000 0 


1: At present, which individuals, groups, of offices are directly involved in day-to-day operations 
of this program? Check all that apply. 

Program Director 

Career Office 

Student affairs office 

Student employment office 

Student financial aid office 

Faculty at department level 

Faculty at college or division level 

President or Provost office 

Business and finance office 

Counsel 

Research office 

Development office 

Human Resources office 

Institutional Office, including assessment and accreditation 

Employer(s) taking student placements 

Employer(s) participating in advisory capacities 

Sponsor/ donor 

SUNY Administration 

Federal, state, regional, local agency 

Other (please specify) 


ooo0ooo0o0o0o0o0q0o0qgo0g00qg 0g 0 0000 
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16. Which individuals, groups, or offices are directly involved in planning and oversight of this 
program? Check all that apply. 
o Program Director 

Career Office 

Student affairs office 

Student employment office 

Student financial aid office 

Faculty at department level 

Faculty at college or division level 

President or Provost office 

Business and finance office 

Counsel 

Research office 

Development office 

Human Resources office 

Institutional Office, including assessment and accreditation 

Employer(s) taking student placements 

Employer(s) participating in advisory capacities 

Sponsor/ donor 

SUNY Administration 

Federal, state, regional, local agency 

Other (please specify) 


oo0o0o0o0o0o0o0o0o0oqoqgoooogoog oo 8 


I, What/who is the most influential individual, group, or office to sustain this program on this 
campus? Check all that apply. 

Program Director 

Career Office 

Student affairs office 

Student employment office 

Student financial aid office 

Faculty at department level 

Faculty at college or division level 

President or Provost office 

Business and finance office 

Counsel 

Research office 

Development office 

Human Resources office 

Institutional Office, including assessment and accreditation 

Employer(s) taking student placements 

Employer(s) participating in advisory capacities 

Sponsor/ donor 

SUNY Administration 

Federal, state, regional, local agency 

Other (please specify) 


oo0o0o0o0o0o0oo0o0oqgoogooooooo0o oO 


18. Now, looking again at the list of influential individual/groups/ offices, which ONE would you 
consider to be the MOST influential to sustain this program on this campus? Check one. 
o Program Director 
o Career Office 
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Student affairs office 

Student employment office 

Student financial aid office 

Faculty at department level 

Faculty at college or division level 

President or Provost office 

Business and finance office 

Counsel 

Research office 

Development office 

Human Resources office 

Institutional Office, including assessment and accreditation 
Employer(s) taking student placements 
Employer(s) participating in advisory capacities 
Sponsor/ donor 

SUNY Administration 

Federal, state, regional, local agency 

Other (please specify) 


ooo0o0o0o0o0go0g00g00g00 00000 0 


1. What/who is the most influential individual, group, or office to expand this program on this 
campus? Check all that apply. 

Program Director 

Career Office 

Student affairs office 

Student employment office 

Student financial aid office 

Faculty at department level 

Faculty at college or division level 

President or Provost office 

Business and finance office 

Counsel 

Research office 

Development office 

Human Resources office 

Institutional Office, including assessment and accreditation 

Employer(s) taking student placements 

Employer(s) participating in advisory capacities 

Sponsor/ donor 

SUNY Administration 

Federal, state, regional, local agency 

Other (please specify) 


ooo0oo0o0o0o0o0o0q0o0q0o0qgo0g0 0g 0 00000 


20. Now, looking again at the list of influential individual/ groups/ offices, which ONE would you 
consider to be the MOST influential to expand this program on this campus? Check one. 

Program Director 

Career Office 

Student affairs office 

Student employment office 

Student financial aid office 

Faculty at department level 


oo0o0o000 
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Faculty at college or division level 

President or Provost office 

Business and finance office 

Counsel 

Research office 

Development office 

Human Resources office 

Institutional Office, including assessment and accreditation 
Employer(s) taking student placements 
Employer(s) participating in advisory capacities 
Sponsor/ donor 

SUNY Administration 

Federal, state, regional, local agency 

Other (please specify) 


oo0oo0o0o0o0o0g0o0g00cg00 000 


21, In your view, what might prevent expansion of this program to more students on this campus? 

Check all that apply. 

o The structure of this program applies to a particular field 

o Faculty in some departments or some colleges or divisions do not support this type of 
applied learning program because there is limited room in the curriculum for it 

o Faculty in some departments or some colleges or divisions do not support this type of 
applied learning program because the knowledge or skills acquired in such a program are 
not believed to contribute substantially to the curriculum objectives 

o Incentives for faculty and staff are insufficient to support the work needed to expand this 
program 

o Employers in sectors served by some graduates do not value the experiences from this type 
of applied learning program in recruitment 

o Students anticipating graduation do not see the benefit of applied learning experiences from 
this type of program for continuation of their studies in another degree program or campus 

o Students anticipating graduation do not see the benefit of applied learning experiences from 
this type of program for employment in their anticipated job or 

o The resources needed to expand this program are not likely to be made available 

o Legal or insurance issues 

o Other (please specify) 


22. Now, looking again at the reasons that might prevent expansion of this program to more 
students on this campus, which ONE would you consider to be the MOST important reason? 
Check one. 

o The structure of this program applies to a particular field 

o Faculty in some departments or some colleges or divisions do not support this type of 
applied learning program because there is limited room in the curriculum for it 

o Faculty in some departments or some colleges or divisions do not support this type of 
applied learning program because the knowledge or skills acquired in such a program are 
not believed to contribute substantially to the curriculum objectives 

o Incentives for faculty and staff are insufficient to support the work needed to expand this 
program 

o Employers in sectors served by some graduates do not value the experiences from this type 
of applied learning program in recruitment 

o Students anticipating graduation do not see the benefit of applied learning experiences from 
this type of program for continuation of their studies in another degree program or campus 
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o Students anticipating graduation do not see the benefit of applied learning experiences from 
this type of program for employment in their anticipated job or 

o The resources needed to expand this program are not likely to be made available 

o Legal or insurance issues 

o Other (please specify) 


20: In your view, what might prevent expansion of this type of program to more students on 

campuses elsewhere in SUNY? Check all that apply. 

o The profiles of academic programs at other campuses are different than what we offer 

o The profiles of students at other campuses are different than at my campus 

o Faculty interest in this type of applied learning program at other campuses is weaker than at 
my campus 

o Student interest in this type of applied learning program at other campuses is weaker than at 
my campus 

o Connections with employers at other campuses are not as strong as at my campus 

o Resources in support of this type of applied learning are less than at my campus 

o Other (please specify) 


24. Now, looking again at the reasons that might prevent expansion of this type of program to 
more students on campuses elsewhere in SUNY, which ONE would you consider to be the 
MOST important reason? Check one. 

o The profiles of academic programs at other campuses are different than what we offer 

o The profiles of students at other campuses are different than at my campus 

o Faculty interest in this type of applied learning program at other campuses is weaker than 
at my campus 

o Student interest in this type of applied learning program at other campuses is weaker than at 
my campus 

o Connections with employers at other campuses are not as strong as at my campus 

o Resources in support of this type of applied learning are less than at my campus 

o Other (please specify) 


20: Please provide any additional thoughts or comments on the development of applied learning 
on your campus. 
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Appendix B: Procedure and Detailed Results of Retention and 
Graduation Effects Using Propensity Score Matching 


We used propensity score matching to obtain estimates of the average treatment effects for the 
treated (ATT), i.e., taking at least one internship. 


Retention 


First, we used all student characteristics in the logistic regression (these are shown in Appendix C, 
Tables 3.8 and 3.9) to predict the propensity of taking at least one internship by the end of the third and 
fourth academic years for each student. Our check of common support shows that the common sup- 
port condition appears to pose no barrier to the analysis, since the ranges of propensity scores for the 
treatment and control groups largely overlapped. In this step, we did not exclude cases from the treat - 
ment group (i.e., internship takers in any year to the year of retention) that lie outside of the range of 
common support, i.e., those cases with propensity scores higher than the maximum value of the con- 
trol group. Had we done so, we may have lost matches at the boundaries of the common support and 
so greatly reduce the number of cases for the analysis.1% See Figures 3.2 and 3.3 below. 

Next, we used a radius caliper of 0.01 to match each internship-taker with any control observations 
within +/- 0.01 of the internship-taker’s propensity score. This strategy helps to avoid the idiosyncratic 
matches that might result from simple one-to-one nearest neighbor matching. Only one case was ex- 
cluded from the control group for both the estimations for third and fourth year retention. 

Then, each individual remaining in the treatment group (taking at least one internship) was 
matched with an individual from the control group whose propensity for taking at least one internship 
was closest to the treated individual. After matching, our treated and control group observations are 
well-balanced on the observable characteristics. As shown in Figures 3.6 and 3.7, the standardized bias 
on the observable background characteristics, i.e., the differences in means between treated and un- 
treated cases expressed in terms of standard deviations, have been reduced, mostly to below 5 percent 
levels. These levels are considered acceptable.1% 

We also did a sensitivity test to estimate how strongly an unobserved attribute or characteristic 
must influence the selection process to eliminate the estimated treatment effect of internship taking on 
the third year retention. Since our outcomes are binary, we followed Becker and Caliendo!” to do this 
test. Results show that, for retention at the end of the third year, the ATT estimation would become in- 
significant at the .10 level only if there was an unobservable characteristic or attribute that would in- 
crease the odds of internship taking by 3.5 times. The relevant statistics are provided in Table 3.12. 
These results suggest the estimation of treatment effects — internship taking on retention — using pro- 
pensity score matching is insensitive to unobservable positive selection. 


Graduation 

Again, we first used all student characteristics in the logistic regression (these are shown in Appen- 
dix C, Table 3.10) to predict the propensity of taking at least one internship by the end of the Spring se- 
mester of the fourth academic year for each student. Our check of common support shows, again, that 
the common support condition appears to pose no barrier to the analysis since the ranges of propensity 
scores for the treatment and control groups largely overlapped. See Figure 3.4. 

Next, we used a radius caliper of 0.01 to match each internship-taker with any control observations 
within +/- 0.01 of the internship-taker’s propensity score. Only one case was excluded from the control 
group. 

Then, each individual remaining in the treatment group (taking at least one internship) group was 
matched with an individual from the control group whose propensity of internship taking is closest to 
the treated individual. After matching, our treated and control group observations are well-balanced 
on the observable characteristics. See Figure 3.7. 
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Again, we carried out sensitivity analyses. The results show that for graduation by the end of the 
Spring semester of the fourth year, the ATT estimation would become insignificant at the .10 level only 
if there was an unobservable characteristic or attribute that would increase the odds of internship tak- 
ing by .4 times. This suggests that the estimation is somewhat sensitive to unobservable positive selec- 
tion. The relevant statistics are provided in Table 3.13. 
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Appendix C: Tables and Figures for the Pilot Study 


MIE-| 0) (ecu lem BY aaCoye lel olalcem-lare moxexed(ol-\ere) ale) tal (em Og al-Le-Ceuccla Chi (ex-) 
of Students in the Pilot Stud 


Number of Percentage of 
Characteristics cases sample 
Cohort 
Fall 2006 2,708 32.4 
Fall 2007 2,760 oo. 
Fall 2008 2,880 34.5 
Gender 
Female 3,941 47.2 
Male 4,407 52.8 
Ethnicity 
American Indian/ Alaska 
Native 19 0.2 
Asian DfT 273 
Black/ African American 486 5.8 
Hispanic of any race 668 8.0 
Non-Resident Alien 436 oe 
Race and Ethnicity unknown 1,733 20.8 
White 2,729 g2./ 
Tuition residency in first term 
Foreign not U.S. 355 4.3 
New York 7,308 87.5 
Out-of-state U.S 685 8.2 
Ever Offered Pell (as proxy to 
family resourcefulness) 
No 5,004 099 
Yes 3,344 40.1 
Overall Sample 8,348 100.0 


Ie | 0) (-ec w4am DY -X-Yed | oL WAV(2 Mo} C14 t} a exo) a OLe) | (-Ye [-M ad d=) of 1K Ve [al st-t-) 


Number of 
Observations Mean _ Std. Dev. Min Max 
SAT Math Score 7715 615.0 73.8 370 800 
SAT Critical Reading Score 7716 562.8 82.4 210 800 
High School GPA 8097 90.2 45 71 99.9 
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Table 3.3. Number of Internships 
Students Took 


N of 
Internships Students Percent 
1 638 44.46 
Z 526 36.66 
3 178 12.40 
4 <8, 3.69 
5 25 1.74 
6 11 0.77 
7 3 0.21 
9 1 0.07 
Total 1,435 100 


Milo) (-Wecwr- Ma lal claeat-dall om -Ldlare Mm on’a Bx-Vaaveyele-l ola) (om-lale motored (ol-Lexe)aco)arl (em =f-(eq.ce] cele lire 


At least one Internship 
Demographic and Socioeconomic Background No Yes 
Gender*** Female 77.59 22.41 
Male 87.47 12.53 
Ethnicity*** American Indian/Alaskan Native 89.47 10.53 
Asian 80.90 19.10 
Black/African American 76.13 23.87 
Hispanic of any race 80.24 19.76 
Non-Resident Alien 86.70 13.30 
Race and Ethnicity Unknown 82.69 17.31 
White 85.64 14.36 
Tuition Residency Foreign not U.S. 87.04 12.96 
in First Term* New York 82.70 17.30 
Out-of-state U.S. 81.75 18.25 
Ever Offered Pell No 83.09 16.91 
Yes 82.39 17.61 
Total 82.81 17.19 
*p<0.10, **p<0.05, **p<0.01 


Table 3.5. Differences in Academic Performance Means by Internship Taking 


Academic 
Performance Group At Least Once Never | Combined Difference 
SAT Math N 1329 6386 7715 
Mean 604.85 617.16 615.04 =12.32*** 

SAT Critical N 1353 6363 7716 

Reading Mean 559.51 563.50 562.80 -3.98* 
High School N 1430 6667 8097 
GPA Mean 90.34 90.13 90.17 0.21 
CollegeGPA- N 1435 6913 8348 

1st Term Mean 3.00 272. 2.77 O27 

Note: Difference is At Least Once - Never 

*p<0.10, **p<0.05, **p<0.01 
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Wie-1o)(-WecMoMm NCY-J01 | Moya atom mole lt) i (om aX-Cola-t-t-J ola mcolmlaiccairvall omic dale| 


Regression Odds Predicted Margina 
Coefficient Ratio Probabilities Effects 
(Log-odds) | (OR) 
Intercept -1.62*** 0.20*** 0.165*** 
Male -0.61*** 0.54*** 0.097*** -0.068*** 
Ethnicity 
American Indian/Alaskan Native -0.33 0.72 0.125 -0.04 
Asian 0.46*** 1.58*** 0.239*** 0.073*** 
Black/African American 0.53*** 1.69*** 0.251*** 0.086*** 
Hispanic of any race 0.34*** 1.41*** O.213*** 0.053*** 
Non-Resident Alien 0.05 1.05 0171" 0.006 
Race and Ethnicity Unknown 0.24*** 1.27" 0.200*** 0.035*** 
Tuition Residency in First Term 
Out-of-state U.S. 0.02 1.02 0.167*** 0.002 
Foreign not U.S. -0.27 0.76 0.131*** -0.034 
Offered Pell -0.13* 0.88* 0.149%** -0.017* 
SAT Math (Standardized) -0.15*** 0.86*** 0.145*** -0.021*** 
SAT Critical Reading (Standardized) -0.03 0.97 0.161*** -0.004 
High School GPA (Standardized) -0.08** 0.93** 0.155*** -0.011** 
College GPA - 1st Term (Standardized) 0.44*** pbk 0.234*** 0.060*** 
Entering Cohort 
2007 0.04 1.04 O17" 0.006 
2008 0.21" 123°" 0.196*** 0.031" 
*p<0.10, *p<0.05, **p<0.01 


Table 3.7. Internship Taking and Retention by Academic Year — A “Flow” Model 


Academic 
Year Intern Status Entered Number Retained Number Retention Rate 
At least once 55 53: 96.36 
i Never 8,293 7 A87 90.28 
Total 8,348 7,540 90.32 
At least once 279 272 97.49 
ot Never 7,261 6,487 89.34 
Total 7,540 6,759 89.64 
At least once 730 728 99.73 
out Never 6,025 5,679 94.26 
Total 6,755 6,407 94.85 
At least once 1,250 1,226 98.08 
40 Never 5,025 4,796 95.44 
Total 6,275 6,022 95.97 
At least once 388 365 94.07 
5s Never 1,878 1,678 89.35 
Total 2,266 2,043 90.16 
At least once 69 56 81.16 
6e* Never 522 399 76.44 
Total 591 455 76.99 
*p<0.10, *p<0.05, **p<0.01 
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Table 3.8 Results of the Logistic Regression 


for Retention at the Beginning of the Fourth Year 


Regression Odds Predicted Margina 
Coefficient Ratio Probabilities Effects 
(Log-odds) (OR) 
Intercept 3.40*** 29.94*** 0.968*** 
Take at least one internship by the end 
of Year 3 2.88*** 17.83*** 0.998*** 0.030*** 
Male -0.02 0.98 0.967*** -0.001 
Ethnicity 
American Indian/Alaskan Native 
Asian 0.56*** 7 5* 0.981*** 0.014*** 
Black/African American 0.25 1.28 0.975*** 0.007 
Hispanic of any race 0.44* 1.55* 0.979*** 0.011* 
Non-Resident Alien 0.75* 211" 0.984*** 0.017* 
Race and Ethnicity Unknown 0.20 1.22 0.9738" 0.006 
Tuition Residency in First Term 
Out-of-state U.S. -0.33 0.72 0.955*** -0.012 
Foreign not U.S. -0.62 0.54 0.941*** -0.026 
Offered Pell 0.11 abil 0.971*** 0.003 
SAT Math (Standardized) -0.05 0.95 0.966*** -0.002 
SAT Critical Reading (Standardized) -0.06 0.94 0.966*** -0.002 
High School GPA (Standardized) 0.02 1.02 0.968*** 0.001 
Cumulative GPA by the end of the 
Spring of Year 2 (Standardized) 1.22"*6 3.38*** 0.990*** 0.038*** 
Entering Cohort 
2007 0.02 1.02 0.968*** 0.001 
2008 0.03 1.03 0.969*** 0.001 
Sample Size 5,909 
Note: The variable American Indian/ Alaskan Native predicts retention perfectly, thus was dropped 
and 11 observations not used 
*p<0.10, *p<0.05, **p<0.01 
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Table 3.9. Results of the Logistic Regression 


for Retention at the Beginning of the Fifth Year__ 


Regression Odds Predicted §Margina 
Coefficient | Ratio Probabilities _ Effects 
(Log-odds) (OR) 
Intercept 4.53*** 92.69*** 0.989*** 
Take at least one internship by the end 
of Year 4 0.29 1.34 0.992*** 0.003 
Male 0.20 1.22 0:991*** 0.002 
Ethnicity 
American Indian/Alaskan Native -1.30 0.27 0.962*** -0.027 
Asian 0.16 1.17 O:991*** 0.002 
Black/African American 0.18 1.20 0.991*** 0.002 
Hispanic of any race -0.21 0.81 0.987*** -0.002 
Non-Resident Alien -0.19 0.83 0.987*** -0.002 
Race and Ethnicity Unknown 0.17 1.18 0.991*** 0.002 
Tuition Residency in First Term 
Out-of-state U.S. -0.52* 0.60* 0.982*** -0.007 
Foreign not U.S. 0.96 2.60 0.996*** 0.007 
Offered Pell -0.23 0.80 0.987*** -0.003 
SAT Math (Standardized) -0.13 0.87 0.988*** -0.001 
SAT Critical Reading (Standardized) -0.25*** 0.78*** 0.986*** -0.003** 
High School GPA (Standardized) 0.11 1.12 0.990*** 0.001 
Cumulative GPA by the end of the 
Spring of Year 3 (Standardized) 1.42*** 4.14"* 0.997*** 0.015*** 
Entering Cohort 
2007 -0.73*** 0.48*** 0.978*** -0.011*** 
2008 -0.43** 0.65** 0.984*** -0.006* 
*p<0.10, **p<0.05, **p<0.01 
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Miele) (ecw CON A X-X-10] | Woy mm al Mole [eda (em nXcVe fu -t-t-J(e) a mcolmCre-lolUr-liCola mms) lalate Mm melel aia im d-\-10 


Regression Odds Predicted Marginal 
Coefficient Ratio Probabilities Effects 
(Log-odds) (OR) 
Intercept 0.07 1.08 0.518*** 
Took at least one internship by the end 
of the Spring Semester of Year 4 0.574" 1.76*** 0.655*** 0.136*** 
Male -0.46*** 0.63*** 0.404*** -0.115*** 
Ethnicity 
American Indian/Alaskan Native -1.08 0.34 0.268* -0.251* 
Asian 0.24*** 1.27% 0.57755" 0.059*** 
Black/ African American 0.43*** 1.54*** 0.624*** (0:105** 
Hispanic of any race -0.06 0.94. 0.504*** -0.015 
Non-Resident Alien 0.00 1.00 0.519"** 0.000 
Race and Ethnicity Unknown 0.07 1.07 0.536*** 0.018 
Tuition Residency in First Term 
Out-of-state U.S. 0.38*** 1.47" 0.613*** 0.094*** 
Foreign not U.S. 0.05 1.05 0.532*** 0.013 
Offered Pell -0.16** 0.85** 0.479*** -0.040** 
SAT Math (Standardized) 0.07* 1.07* 0.536*** 0.018* 
SAT Critical Reading (Standardized) -0.12*** 0.89*** 0.490*** -0.029*** 
High School GPA (Standardized) -0.08** 0.93** 0.500*** -0.019** 
Cumulative GPA by the end of the 
Spring Semester of Year 4 
(Standardized) 1.30"* 3.68*** 0.799*** 0.325*** 
Entering Cohort 
2007 0.09 1.10 0.541*** 0.023 
2008 0.16** 118" 0.559*** 0.040** 
*p<0.10, **p<0.05, **p<0.01 


Table. 3.11. Effects (ATT) of Internship-Taking 


oy aM a<clCclalicedamee lace merece lur-Liceyal 


Outcomes ATT 
Retention at the End of the 3rd Year 0.028*** 
Retention at the End of the 4th Year 0.004 
Graduation in Spring of the 4th Year 0.095*** 


ATT: Average Treatment Effects for the Treated. 
*p<0.10, *p<0.05, **p<0.01 
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Table 3.12. Sensitivity Test: Mantel-Haenszel (1959) 


Bounds for Retention at the End of the Third Year 
Gamma Q mh+ Qmh- p_mh+_ p_mh 
1 3.814 3.814 0.000 0.000 
1.5 2.989 4.792 0.001 0.000 
2 2.475 5.597 0.007 0.000 
2.5 2.112 6.298 0.017 0.000 
3 1.833 6.926 0.033 0.000 
3.5 1.610 7.501 0.054 0.000 
4 1.424 8.034 0.077 0.000 
45 1.265 8.534 0.103 0.000 
5 1.126 9.005 0.130 0.000 
5.5 1.004 9.452 0.158 0.000 
6 0.894 9.879 0.186 0.000 
Gamma : odds of differential assignment due to unobserved factors 
Q_mht+ : Mantel-Haenszel statistic (assumption: overestimation of treatment effect) 
Q_mh- : Mantel-Haenszel statistic (assumption: underestimation of treatment effect) 
p_mbh+ : significance level (assumption: overestimation of treatment effect) 


p_mbh- : significance level (assumption: underestimation of treatment effect) 


Table 3.13. Sensitivity Test: Mantel-Haenszel (1959) 


Bounds for Graduation, Spring of the Fourth Year 
Gamma Qmh+ Qmh-- p_mht _ p_mh- 
1 4.081 4.081 0.000 0.000 
1.1 3,131 5.036 0.001 0.000 
di2 2.265 5.910 0.012 0.000 
13 1.469 6.717 0.071 0.000 
1.4 0.733 7.467 0.232 0.000 
1.5 0.048 8.168 0.481 0.000 
Gamma : odds of differential assignment due to unobserved factors 
Q_mht+ : Mantel-Haenszel statistic (assumption: overestimation of treatment effect) 
Q_mbh- : Mantel-Haenszel statistic (assumption: underestimation of treatment effect) 
p_mh+ : significance level (assumption: overestimation of treatment effect) 


p_mh- : significance level (assumption: underestimation of treatment effect) 
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Figure 3.1. Number of Students Taking Internships by Year and Term 
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Figure 3.3. Common Support — Retention at the 
End of the Fourth Year 
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Figure 3.5. Bias Reduction Graph — Retention at the 
End of the Third Year 
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Figure 3.6. Bias Reduction Graph — Retention at the 


End of the Fourth Year 
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Figure 3.7. Bias Reduction Graph — Graduation, 


Spring of the Fourth Year 
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868-81; Debbie Easterling and Frederic Rudell, “Rationale, Benefits and Methods of Service Learning in 
Marketing Education,” Journal of Education for Business 73, 1 (1997): 58-61; Robert Shumer, “Learning from 
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Undergraduates Are Affected by Service Participation,” Journal of College Student Development 39, 3 (1998): 
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4 SUNY Works Grant Proposal. Submitted to Lumina Foundation for Education, August 18, 2010. 
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Swati Desai and Katie Zuber, “SUNY Works: A Baseline Report,” Delivered to SUNY System 
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8 | We had conversations with Nancy Hoffman and Amy Lloyd, Jobs for the Future; Dennis Jones and Patrick 
Kelly, National Center for Higher Education Management Systems; Steven Robbins, Samuel Rikoon, and 
Jennifer Lentini, Educational Testing Service; Sandra Gregerman, University of Michigan; Richard Freeland, 
commissioner of higher education, Massachusetts, and former president, Northeastern University; Peter 
Smith, Kaplan Higher Education and former president, California State University at Monterey Bay; and 
Earl Hale, former executive director , Washington State Board for Community and Technical Colleges. 


9 “SUNY Applied Learning Workshop” in Syracuse, NY, September 18-19; Rutgers/Federal Reserve Banks of 
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Brunswick, NJ, October 15-17; Center for Analysis of Postsecondary Education and Employment (CAPSEE), 
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10 Dennis Jones, president, National Center for Higher Education Management Systems, provided advice and 
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York’s Workforce Needs,” The Best of Co-Op: A Guide to the Leading Colleges & Employers (Albany, NY: State 
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